-- Logs begin at Thu 2022-07-14 13:32:07 UTC, end at Thu 2022-07-14 14:55:12 UTC. -- Jul 14 14:07:46.638588 dv-cinder-799871-30-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 14:07:46.650281 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 14:07:46 2022] *** Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: nodename: dv-cinder-799871-30-iscsi Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: machine: x86_64 Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: clock source: unix Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: pcre jit disabled Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: detected number of CPU cores: 4 Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: current working directory: / Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: detected binary path: /usr/bin/uwsgi-core Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: your processes number limit is 47680 Jul 14 14:07:46.652946 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: your memory page size is 4096 bytes Jul 14 14:07:46.653727 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: detected max file descriptor number: 2048 Jul 14 14:07:46.653727 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: lock engine: pthread robust mutexes Jul 14 14:07:46.653727 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: thunder lock: enabled Jul 14 14:07:46.653727 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 14:07:46.653727 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 14:07:46.696267 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: Python main interpreter initialized at 0x563f74eb59c0 Jul 14 14:07:46.696267 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: python threads support enabled Jul 14 14:07:46.696267 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: your server socket listen backlog is limited to 100 connections Jul 14 14:07:46.696267 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: your mercy for graceful operations on workers is 90 seconds Jul 14 14:07:46.696620 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 14:07:46.696692 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: *** Operational MODE: preforking *** Jul 14 14:07:46.696956 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: *** uWSGI is running in multiple interpreter mode *** Jul 14 14:07:46.696956 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: spawned uWSGI master process (pid: 80315) Jul 14 14:07:46.697158 dv-cinder-799871-30-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 14:07:46.701809 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: spawned uWSGI worker 1 (pid: 80327, cores: 1) Jul 14 14:07:46.701809 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: spawned uWSGI worker 2 (pid: 80328, cores: 1) Jul 14 14:07:46.701809 dv-cinder-799871-30-iscsi devstack@keystone.service[80315]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 14:07:52.226990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 14:07:52.226990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: from cryptography.utils import int_from_bytes Jul 14 14:07:52.227797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 14:07:52.227797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: from cryptography.utils import int_from_bytes Jul 14 14:07:52.227918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 14:07:52.227918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: from cryptography.utils import int_from_bytes Jul 14 14:07:52.227918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 14:07:52.227918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: from cryptography.utils import int_from_bytes Jul 14 14:07:52.290642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 14:07:52.290828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 14:07:52.290923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 14:07:52.290998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 14:07:52.291067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] command line args: [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 14:07:52.291130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] command line args: [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 14:07:52.291275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 14:07:52.291362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 14:07:52.291441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ================================================================================ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 14:07:52.291516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ================================================================================ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 14:07:52.291716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] admin_token = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.291799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] admin_token = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.291964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_dir = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_dir = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_source = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_source = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] debug = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] debug = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.292969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.293894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_config_append = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_config_append = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_dir = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_dir = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.294931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.295926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.296968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.297933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.298134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.298226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.298627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.298732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.298968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] publish_errors = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] publish_errors = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.299941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] standard_threads = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] standard_threads = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.300925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_journal = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_journal = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_json = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_json = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_stderr = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_stderr = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_syslog = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_syslog = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 14:07:52.301931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.302958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.303928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304299 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.304920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.305908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.306985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.307158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.307738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.307974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.308941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.309935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.310723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.311972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.312993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.313986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.314945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.315842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.316950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.317933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.318984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.319949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.320942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.321933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.322960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.323631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.323711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.323781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.323928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.324960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.325918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.326621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.327861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.external = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.external = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.328918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.password = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.password = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.329874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.token = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.token = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.330913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.331879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.332958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.333088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.333215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.333904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.333994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.334947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.335998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.336877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.337939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.338952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.339898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.340859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.341010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.341098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.341248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.341496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.341740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.342912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.343925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.344643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.345825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.346076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.346329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.346563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.346800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.348995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349739 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.349978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.350052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.350124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.350202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.350275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.350345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.352951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.353982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.354927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.355958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.356984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.357973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.358971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.359919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.360973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.361957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.362948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.363934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.364913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.365888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.366875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.367848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.368972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.369058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.369223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.369302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.369460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.369650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.370931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.371926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.372961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.373942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.374881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.375875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.376988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.377981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.378939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 14:07:52.379750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.379994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.380240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.380481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.380721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.380960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.381201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.381449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.381689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.381959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.382199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.382451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.382690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.382938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.383172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.383417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.383656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 14:07:52.383831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 14:07:52.411820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.412052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.412283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.414016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.414256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.414488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.420824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.422950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.450301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.450533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.452840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.453075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.593691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.595229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.644769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.645013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.647742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.647982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.710551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.710777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.710870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.711870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.737945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.738841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.750187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.750614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.762996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.763974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.764161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.764247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.764376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.764458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.764595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 14:07:52.781580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.781816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 14:07:52.782007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.782224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 14:07:52.782388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.782593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 14:07:52.782790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.782874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.783001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 14:07:52.783120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.783365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.783456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.783622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.783764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.784035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.784297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.795042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.795320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.795587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.795865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.795977 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.796261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.796534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.796819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.804106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.805260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.808011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.808294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.808577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.808860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.809221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.809955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.810836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.811026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.811299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.811574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.811846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.837425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.839351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.839618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.839903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.841044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.841331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.845295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.845515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.845815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.849020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.849462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.849871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.853373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.853657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.858771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.859058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.859344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.859540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.859633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.859835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.859952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.860149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.860419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.860689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.864357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.864648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.864932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.865215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.865493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.865789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.866065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.866335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.879294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.879574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.884280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.885098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.885408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.887933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.888230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.890104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.893375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.893929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.894231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.897001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.899384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.900656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.901474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.901784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.902075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.902350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.902630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.903555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.907338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.907639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.907935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.908451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.908550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.908793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.922224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.922587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.923061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.923525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.929248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.929544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.929840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.930114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.932304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.934036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.934326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.937516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.937822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.938096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.938180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.938378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.938722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.939889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.940169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.943408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.943696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.943983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.944272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.944613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.952260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.952541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.952824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.953120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.953397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.953684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.953981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.958302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.958590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.958876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.959169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.959464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.959742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.960018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.973716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.976996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.979960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.980383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.982006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.983786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.984266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.984548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.984836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.987252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.988862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.990665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.990955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.991252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.992227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.993867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:52.997170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.997459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.997745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.998055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:52.998655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.000290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:53.003646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.003943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.004229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.004507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.007251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:53.007539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.007819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.013781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:53.014084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.014368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.015773 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:53.016054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.016608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.016877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.017165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.017444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.017719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.018112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.018383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.018654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.018916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.019193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.019460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.022369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 14:07:53.022655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.022937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.023217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.023491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.023782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.024060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.024335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.024711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.024986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.025261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.025534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.025820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.026627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 14:07:53.058332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563f74eb59c0 pid: 80328 (default app) Jul 14 14:07:53.061796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a91a9-15ed-455a-80b5-724726492870 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:07:53.062104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a91a9-15ed-455a-80b5-724726492870 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:07:53.062394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a91a9-15ed-455a-80b5-724726492870 None None] PATH_INFO: `/v3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:07:53.065224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563f74eb59c0 pid: 80327 (default app) Jul 14 14:07:53.066612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1/1] 10.222.0.93 () {50 vars in 791 bytes} [Thu Jul 14 14:07:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:01.683488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:01.684095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:01.684095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:01.688597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1/2] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:01.694310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:01.694769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:01.694861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:01.765877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: INFO dbcounter [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] Registered counter for database keystone Jul 14 14:08:01.787048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG oslo_db.sqlalchemy.engines [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 14:08:01.793963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writer thread running {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 14:08:01.972517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG passlib.handlers.bcrypt [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80328) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 14:08:01.972842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG passlib.handlers.bcrypt [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80328) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 14:08:02.028724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:02.030376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:02.152788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 2/3] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 14:08:02.325426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:02.399123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: INFO dbcounter [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] Registered counter for database keystone Jul 14 14:08:02.418912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG oslo_db.sqlalchemy.engines [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80327) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 14:08:02.425610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writer thread running {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 14:08:02.550951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dedbdc-5af3-45a6-881e-5da006b97dd3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:02.555607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:02.556302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:02.556594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:02.556880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:02.565116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:02.617317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING keystone.common.rbac_enforcer.enforcer [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 14:08:02.617598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fb91f-755e-4955-a955-0c6255331fd3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:02.628062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 2/4] 10.222.0.93 () {58 vars in 1324 bytes} [Thu Jul 14 14:08:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:08:02.634855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:02.649390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d157b-b63c-4d31-b55b-3a6dbf53e029 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:02.654054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:02.655184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:02.655537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:02.656855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] PATH_INFO: `/v3/projects/df24a39f5f0a46c8b40d27ec05417d32` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:02.658516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] RBAC: Authorizing `identity:get_project(project_id=df24a39f5f0a46c8b40d27ec05417d32)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:02.711652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING keystone.common.rbac_enforcer.enforcer [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 14:08:02.712208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b18ff-1a70-4f54-8214-433c34d2eda1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:02.716553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 3/5] 10.222.0.93 () {56 vars in 1207 bytes} [Thu Jul 14 14:08:02 2022] GET /identity/v3/projects/df24a39f5f0a46c8b40d27ec05417d32 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:04.734355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ef2c-39cb-45bd-9c86-e03a158e0570 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:04.734909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ef2c-39cb-45bd-9c86-e03a158e0570 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:04.734909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ef2c-39cb-45bd-9c86-e03a158e0570 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:04.739188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 3/6] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:04.744539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:04.744949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:04.745103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:04.764083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:04.765590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:04.773326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 4/7] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 14:08:04.795839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4ef2c-39cb-45bd-9c86-e03a158e0570 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:04.807844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4ef2c-39cb-45bd-9c86-e03a158e0570 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:04.812042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:04.812680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:04.812993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:04.813436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:04.821190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:04.823510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8769c-1bfe-4eae-8ef9-421d6a8276b1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:04.832699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 4/8] 10.222.0.93 () {58 vars in 1324 bytes} [Thu Jul 14 14:08:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 14:08:04.838689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:04.847735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a257e-df82-4602-8fe9-f3414672e77b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:04.852071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:04.852926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:04.853209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:04.853483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:04.854917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:04.856726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7039a58-ee86-4d0f-8d2d-e4f544856dd3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:04.858325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 5/9] 10.222.0.93 () {56 vars in 1198 bytes} [Thu Jul 14 14:08:04 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:06.935041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08eab79-660f-4cac-927a-65dfdf0536f0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:06.935596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08eab79-660f-4cac-927a-65dfdf0536f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:06.935596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08eab79-660f-4cac-927a-65dfdf0536f0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:06.936937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 5/10] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:06.942448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:06.942894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:06.943020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:06.960476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:06.961679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:07.031814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e2f8-66b3-4cba-a542-428ff4ba1cf7 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.032201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e2f8-66b3-4cba-a542-428ff4ba1cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.032491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e2f8-66b3-4cba-a542-428ff4ba1cf7 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.033874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 6/11] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:07.038790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.039162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.039508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.042379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 6/12] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:07.066919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.081195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG passlib.handlers.bcrypt [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80327) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 14:08:07.081621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG passlib.handlers.bcrypt [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80327) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 14:08:07.092858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.127574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:07.129190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:07.172853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac53ca-453e-4880-a3e5-c9644633d6d5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.207830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.208558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.208847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.209130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.221037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:07.230758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 7/13] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:07.232426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:07.233505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:07.234410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 7/14] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:07.239675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.253518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.335755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.343061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771f83b-bf8f-4747-bd37-09993ccc67b0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.345373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.354738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c63cb-7028-4821-94d9-b083a75950cc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.379262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.380200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.380490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.380774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.381701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:07.382158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.383707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.390064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.390705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.390991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.391270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.395830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 8/15] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:07.401432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.458698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.460101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.504642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295118-3a71-4cc0-8a0b-2c5df5ea659a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.540341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c826f-caf5-491a-923d-d302174a552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.541207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c826f-caf5-491a-923d-d302174a552b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.541489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c826f-caf5-491a-923d-d302174a552b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.541854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c826f-caf5-491a-923d-d302174a552b None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.543381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c826f-caf5-491a-923d-d302174a552b None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.544856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c826f-caf5-491a-923d-d302174a552b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.545619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.546520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 9/16] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:07.551922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c826f-caf5-491a-923d-d302174a552b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.564860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 8/17] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:08:07.570444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.586873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.653784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c826f-caf5-491a-923d-d302174a552b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.672535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc39fa3-4d99-42ae-897d-07cbe1c39a12 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.690749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.691538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.691855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.692136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.693351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.694858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.696497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 10/18] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:07.702557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.711128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.711865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.712152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.712464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.723538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.724654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:08:07.743646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:07.744512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:07.745497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:08:07.745497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:08:07.745497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:08:07.745497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:07.745497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 9/19] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:07.751147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:07.785934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.803705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae735010-981b-469d-8a72-773e2cb09e9b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.805230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:07.837995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.838665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.839132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.839583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.840693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.domain.id=default, user.id=2be999d55fd24e54ae049877c17b8c43) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:07.841008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.842473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1d01c-b360-4f95-8cc1-5cdb3b5d258d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.850509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 11/20] 10.222.0.93 () {56 vars in 1322 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/role_assignments?user.id=2be999d55fd24e54ae049877c17b8c43&scope.domain.id=default&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:07.851940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeae641-7ba4-4e6e-8317-edff96d02c2e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:07.891247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:07.891923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:07.892214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:07.892497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:07.893250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:07.893562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:07.895035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56abcba-4168-46a8-a2e6-5201456576e6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:07.915680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 10/21] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:10.132688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567109b-daaa-44ad-8346-38a185a497ea None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.133251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567109b-daaa-44ad-8346-38a185a497ea None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.133858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567109b-daaa-44ad-8346-38a185a497ea None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.135237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 12/22] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:10.140857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109308b-94c1-4db2-95b1-844e29b72fd7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.141301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109308b-94c1-4db2-95b1-844e29b72fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.141471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109308b-94c1-4db2-95b1-844e29b72fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.174878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4109308b-94c1-4db2-95b1-844e29b72fd7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:10.176104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4109308b-94c1-4db2-95b1-844e29b72fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:10.265181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 11/23] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:10.307881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567109b-daaa-44ad-8346-38a185a497ea None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:10.464896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567109b-daaa-44ad-8346-38a185a497ea None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:10.522641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:10.523928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.524429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.524911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.542938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:10.551429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294c0b-84e3-45de-9e31-62a443f70fe9 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.552041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294c0b-84e3-45de-9e31-62a443f70fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.552535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294c0b-84e3-45de-9e31-62a443f70fe9 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.554149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 12/24] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:10.554810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:10.556307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:10.557064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:10.557878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 13/25] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:10.563916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aae6-0fb1-4d30-bd68-980ee0aae59b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.564530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aae6-0fb1-4d30-bd68-980ee0aae59b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.565008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aae6-0fb1-4d30-bd68-980ee0aae59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.565569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:10.599475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d827aae6-0fb1-4d30-bd68-980ee0aae59b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:10.603367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d827aae6-0fb1-4d30-bd68-980ee0aae59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:10.627605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:10.654155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:10.704936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e7f6a-078c-42c8-a537-6dd71faae392 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:10.747321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:10.747997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.748281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.748559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.749321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:10.749628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:10.751133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894e61f-0fd3-43a7-a757-e53299f86221 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:10.759899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 14/26] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:10.761137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c227a1b-a112-4fdb-9e98-78c12d3283cf None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.761629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c227a1b-a112-4fdb-9e98-78c12d3283cf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.762103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c227a1b-a112-4fdb-9e98-78c12d3283cf None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.763511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 15/27] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:10.766056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c227a1b-a112-4fdb-9e98-78c12d3283cf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:10.776835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 13/28] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:10.778413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.778926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.779396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.827650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:10.830942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:10.930224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:10.944352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c227a1b-a112-4fdb-9e98-78c12d3283cf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:10.982049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:10.983329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:10.983805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:10.984260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:10.985735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:10.987531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:10.989828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 16/29] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:10 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:10.990868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.024350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 14/30] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:11.025254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.107158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5df03-2db4-46cf-8be5-13d48ce4f392 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.141811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:11.151340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.152672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.153107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.154929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.158629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71600b70-e7ae-4941-b687-caa1dce2c6d8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.175542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.177197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.194495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.196072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.196500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.196934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.198286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.200160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7e8d204e-687d-4db6-bf17-20fa0e669139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:08:11.200285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:11.201181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:11.202152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:08:11.202152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:08:11.202152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:08:11.202152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:11.202152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 17/31] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:11.203404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18b8f-7dda-43c7-9a47-5f051372baeb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.203702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18b8f-7dda-43c7-9a47-5f051372baeb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.203984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18b8f-7dda-43c7-9a47-5f051372baeb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.204527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 15/32] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:11.205417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.210196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 18/33] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:11.211027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18b8f-7dda-43c7-9a47-5f051372baeb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.229313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:11.308525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d5598-7834-4333-bbdc-7a675ca1c910 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.335809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18b8f-7dda-43c7-9a47-5f051372baeb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.344381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.345351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.345803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.346225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.347119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.348639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.376136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.376809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.377153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.377529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.378402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:11.378723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.380158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd331a2a-6b80-42a5-bf28-8c3a72c37110 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.401418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 19/34] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:11.402974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.403468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.404019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.421172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:11.422394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:11.497974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:11.505084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 20/35] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:11.506282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.547037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 16/36] 10.222.0.93 () {60 vars in 1131 bytes} [Thu Jul 14 14:08:11 2022] POST /identity/v3/roles => generated 219 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:11.548034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.565287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:11.642204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5b98-dda2-45c3-a821-681fff5f4bee None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.698934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.701146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.701431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.701711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] PATH_INFO: `/v3/domains/default/users/2be999d55fd24e54ae049877c17b8c43/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.707351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2be999d55fd24e54ae049877c17b8c43, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.709726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109ab28-4c9a-4966-bb22-3072091e2004 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.720481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b225-a92b-4574-8438-3df413afcb4c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:11.760833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:11.761515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.761858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.762224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.762929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:11.764545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:11.889856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 17/37] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:08:11 2022] POST /identity/v3/projects => generated 301 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:11.889856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:11.942917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 21/38] 10.222.0.93 () {58 vars in 1361 bytes} [Thu Jul 14 14:08:11 2022] PUT /identity/v3/domains/default/users/2be999d55fd24e54ae049877c17b8c43/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:11.945357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020853-094d-45c7-a322-7b9ba4ef3be6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.946914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020853-094d-45c7-a322-7b9ba4ef3be6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.947742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020853-094d-45c7-a322-7b9ba4ef3be6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.953891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 22/39] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:11.959223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20047-2985-41f0-9ae3-de2c809b80cc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:11.959890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20047-2985-41f0-9ae3-de2c809b80cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:11.960378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20047-2985-41f0-9ae3-de2c809b80cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:11.981863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54b20047-2985-41f0-9ae3-de2c809b80cc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:11.983922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54b20047-2985-41f0-9ae3-de2c809b80cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:12.064307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:12.092117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:12.102859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 23/40] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:12.106017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339b89-851d-42ee-b7d9-dd8e242d0494 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.106619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339b89-851d-42ee-b7d9-dd8e242d0494 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.106912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339b89-851d-42ee-b7d9-dd8e242d0494 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.112574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 24/41] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:12.114236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.114547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.114994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.139750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:12.141046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:12.152746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a542-9ede-45f2-bc84-cd7839928d40 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:12.202605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:12.203720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.203956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.204562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.205330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:12.208401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:12.226640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 25/42] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:12.228067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:12.289883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 18/43] 10.222.0.93 () {60 vars in 1131 bytes} [Thu Jul 14 14:08:11 2022] POST /identity/v3/roles => generated 225 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:12.290947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:12.356159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48b19-4ee2-433d-bae8-b2a9fd35ebbd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:12.386296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:12.406513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:12.407456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.407886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.408307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.409398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:12.411532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ba21c-2900-4f4d-a221-f2538499155b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:12.431706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c21b17-57cf-4f97-858d-b9ce3bd7c7c2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:12.461970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:12.474256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:12.475087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.475714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.475996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.477482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:12.479031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:12.480876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 19/44] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:12.482285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 26/45] 10.222.0.93 () {60 vars in 1131 bytes} [Thu Jul 14 14:08:12 2022] POST /identity/v3/roles => generated 223 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:12.486214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:12.614868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9186b-0584-40ec-8c16-a8c58c1058c2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:12.656667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:12.657615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:12.658121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:12.658613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:12.659353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:12.660911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8c961-4d65-4c26-8366-ebbdce5c1026 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:12.743016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 20/46] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:08:12 2022] POST /identity/v3/projects => generated 312 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:14.189151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e990-1a6b-4892-a898-24d198ad92fd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.189691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e990-1a6b-4892-a898-24d198ad92fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.189800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e990-1a6b-4892-a898-24d198ad92fd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.191131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 27/47] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:14.197073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.197480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.197639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.215333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:14.216839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:14.257794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:14.282692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 21/48] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:14.306424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0e990-1a6b-4892-a898-24d198ad92fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:14.408169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0e990-1a6b-4892-a898-24d198ad92fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:14.442063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:14.442857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.443153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.443440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.461701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:14.474867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:14.475757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:14.477042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:14.477629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 28/49] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:14.483765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:14.586755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632aa902-a056-4462-9a0c-cc874b1ef216 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:14.621053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:14.621729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.622049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.622333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.623087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:14.623399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:14.625062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:14.637199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 22/50] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:14.642754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:14.684202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:14.746913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467aad-10e7-4100-acc8-004ae2e01e8e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:14.784087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:14.785000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.785284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.785563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.786803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:14.788270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b678e0-6fa0-4b69-b39c-10cd8507ad6c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:14.789969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 29/51] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:14 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:14.795431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:14.804465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aea5b8-073e-4536-91f9-7db542865232 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.805404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aea5b8-073e-4536-91f9-7db542865232 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.805732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aea5b8-073e-4536-91f9-7db542865232 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.807172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 30/52] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:14.812842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.813137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.813421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.840057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:14.910739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80fc5c-1391-4aef-b33b-91378346e1ea None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:14.936600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:14.937784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:14.946860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:14.947660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:14.947951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:14.948236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:14.949430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:14.951159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:14.952794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 23/53] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:14.958630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:15.007011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 31/54] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:15.029934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:15.064139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:15.064361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e418-6869-40b2-9ae5-b2e074fc5fe3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:15.066809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:15.098989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:15.099659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:15.099952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:15.100237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:15.101081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.domain.id=default, user.id=2be999d55fd24e54ae049877c17b8c43) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:15.101922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:15.103534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:15.110896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 24/55] 10.222.0.93 () {56 vars in 1322 bytes} [Thu Jul 14 14:08:14 2022] GET /identity/v3/role_assignments?user.id=2be999d55fd24e54ae049877c17b8c43&scope.domain.id=default&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:15.132468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13266dd0-2674-4fb0-99cc-e0e551a06fec None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:15.171114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:15.171946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:15.172240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:15.172528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:15.173762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:15.175268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07315508-9049-420a-9f42-d854f6338a83 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:15.176957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 32/56] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:15.182911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:15.290619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00556c9-d525-43fa-9d4a-eb2ce2127f28 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:15.326265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:15.326940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:15.327234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:15.327659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:15.328383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:15.329886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011c6d-f7d9-4d1b-91c4-29315532f87d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:15.450246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 25/57] 10.222.0.93 () {60 vars in 1140 bytes} [Thu Jul 14 14:08:15 2022] POST /identity/v3/projects => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:08:17.530840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c76d5-1cc8-44d4-b9d1-d3f220932a9f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:17.531362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c76d5-1cc8-44d4-b9d1-d3f220932a9f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:17.531449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c76d5-1cc8-44d4-b9d1-d3f220932a9f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:17.532786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 33/58] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:17.538152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:17.538641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:17.538810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:17.555564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:17.556778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:17.573065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:17.626253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 26/59] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:17.652218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c76d5-1cc8-44d4-b9d1-d3f220932a9f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:17.756143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c76d5-1cc8-44d4-b9d1-d3f220932a9f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:17.760855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:17.796058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:17.796806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:17.797088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:17.797365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:17.798598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:17.800087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5de4-5f99-4442-992f-fa11864c2acb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:17.801715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 34/60] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:17.807782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:17.904346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d25a9-9bb3-45a9-8023-47d7327155dc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:17.937434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:17.938118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:17.938406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:17.938685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:17.939402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:17.940844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93cf05-7061-4826-bd53-8a4ccb946f4e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:18.065996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 27/61] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:17 2022] POST /identity/v3/users => generated 275 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:21.231489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cd3bc-271e-4bda-9831-66d885a60497 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.231998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cd3bc-271e-4bda-9831-66d885a60497 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.232104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cd3bc-271e-4bda-9831-66d885a60497 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.233681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 35/62] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:21.238572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ce679-843b-43d3-84fc-45574264161f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.238999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ce679-843b-43d3-84fc-45574264161f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.239197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ce679-843b-43d3-84fc-45574264161f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.256133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:21.262526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0b7ce679-843b-43d3-84fc-45574264161f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:21.263728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0b7ce679-843b-43d3-84fc-45574264161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:21.353508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 28/63] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:21.373903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d51-0ed3-4914-befa-0552844bb1d4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.374569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d51-0ed3-4914-befa-0552844bb1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.375053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d9220-c407-4307-911a-bceb33ee2f64 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.375312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d51-0ed3-4914-befa-0552844bb1d4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.375439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d9220-c407-4307-911a-bceb33ee2f64 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.375623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d9220-c407-4307-911a-bceb33ee2f64 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.377203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 29/64] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:21.377792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 36/65] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:21.382689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3d51-0ed3-4914-befa-0552844bb1d4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.382811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0f20-f7ab-4a84-8fb0-b2390af0387e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.383069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0f20-f7ab-4a84-8fb0-b2390af0387e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.383352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0f20-f7ab-4a84-8fb0-b2390af0387e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.404691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-066f0f20-f7ab-4a84-8fb0-b2390af0387e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:21.405955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-066f0f20-f7ab-4a84-8fb0-b2390af0387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:21.481313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 30/66] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:21.482917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.483300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.483595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.500067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3d51-0ed3-4914-befa-0552844bb1d4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:21.501444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:21.502796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:21.537990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:21.539260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.539747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.540247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.551451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:21.560077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:21.560922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:21.561903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:21.565035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 37/67] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:21.566219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.571992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:21.582454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 31/68] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:21.583576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.601754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:21.674503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9072-7ce9-4e59-a016-37b007fe5056 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:21.699429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4205f3-0b08-4e63-879a-149fadad00eb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:21.713740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:21.715013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.715569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.716081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.727985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:21.737153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:21.738077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:21.738928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:21.739601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:21.739601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.739601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.739601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.739896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:21.739896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:21.741157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:21.742654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 38/69] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:21.743766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.750698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 32/70] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:21.751670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.821460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:21.845554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:21.855165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865dd8b-be44-41a6-ae2e-551a4f20e583 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:21.869508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242eb0-b6e5-45ba-b779-7a617674d294 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:21.890788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:21.891949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.892457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.893071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.904686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:21.907549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:21.908428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:21.908885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:21.909318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:21.910261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:21.910733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:21.912274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:21.913970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:21.914855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:21.915855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 39/71] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:21.916812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:21.925506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 33/72] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:21.926746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.020734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66388e2-9f43-438f-94e6-3f8bdcfc4332 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.032335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3381c-699a-4a82-804b-bf412df49e95 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.052274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.055252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.056187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.056478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.056769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.058059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.062720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89d747-99d0-4339-a027-6c60755ed0ae None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.062720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 40/73] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.062720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdffd3-6fde-4c2e-a7a9-23de34121e6b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.063611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdffd3-6fde-4c2e-a7a9-23de34121e6b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.063611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdffd3-6fde-4c2e-a7a9-23de34121e6b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.068261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 41/74] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:22.068261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdffd3-6fde-4c2e-a7a9-23de34121e6b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.068461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.069064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.069379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.069957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.070782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:22.071099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.072495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e5af4-96f3-444f-bd0d-24a561a2dffe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.082982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 34/75] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.084411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ccbd-51c8-4148-9057-e0ef0d2c131d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.084711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ccbd-51c8-4148-9057-e0ef0d2c131d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.085803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ccbd-51c8-4148-9057-e0ef0d2c131d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.087848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 35/76] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:22.088680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ccbd-51c8-4148-9057-e0ef0d2c131d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.093990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.176121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdffd3-6fde-4c2e-a7a9-23de34121e6b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.194996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ccbd-51c8-4148-9057-e0ef0d2c131d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.210523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.211398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.211691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.212107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.227663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.229119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f3377-054c-4663-b746-72560b8c1338 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.229864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.230520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.230817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.230934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 42/77] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.231096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.232024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.232465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.232611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.249828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:22.251043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:22.284624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.305298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.306823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f14a7a-c1d6-4cd8-99b3-d5367a6c0a07 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.322753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 43/78] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:22.323617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.335785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.411335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 36/79] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:22.412623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.412916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.413220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.426598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1077-7892-4d5e-a161-e86c2f72a549 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.429578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:22.430807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:22.461717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.462631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.462922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.463204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.464404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.465903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.467557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 44/80] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.468308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.501601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 37/81] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:22.502491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.517489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.555956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.580615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9097b-a671-4345-b603-7e57fb427183 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.605081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667601f-a7d1-4e5c-990c-1073604338e3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.617229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.617924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.618220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.618509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.638604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.639397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.639687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.639962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.641129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.642623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.644415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 38/82] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.645015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.689971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.691373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.726468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.745903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb796c28-a234-4b47-aa55-fc7762122ed8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.765742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.779731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.780587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.780876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.781156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.782384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.783833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:22.785895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 39/83] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:22.786327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.798134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 45/84] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:22.799590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.888071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02a6a3-82b8-455b-86cf-309f869d7491 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.899099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdd412-23f6-4343-bda3-2783f7820313 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:22.922670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.923406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.923759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.924103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.934886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:22.935692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:22.936025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:22.936405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:22.942279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.946788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:22.954832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:22.955993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:22.956873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 46/85] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:22.957597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:22.994015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:22.997806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:22.999348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.061457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c98b-11e4-4f73-b871-e6ef9accb2e6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.098207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 40/86] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:23.098386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.099087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.099282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.099375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.099653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.100415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.102473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.148944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:23.186412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 47/87] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:22 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:23.187650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.212687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b81d-db70-4d40-9783-afa153819c4c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.253304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.253974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.254271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.254580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.255428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=beafb93941304589a5106acd52069473, user.id=2be999d55fd24e54ae049877c17b8c43) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:23.255755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.257805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.265072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 41/88] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/role_assignments?user.id=2be999d55fd24e54ae049877c17b8c43&scope.project.id=beafb93941304589a5106acd52069473&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.265981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.310572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:23.332111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae3bf-2777-4287-9883-2d297eb80e5f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.393441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.394381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.394679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.394968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.396876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.399957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.402520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 48/89] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.403636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.405298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3fd14-f057-496d-9eb2-edd3ab52f28c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.422134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:23.447470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.448152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.448483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.449031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.449814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:23.450132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.451515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.461861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 42/90] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.463239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.536065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec673f2-2f24-410a-a3b4-5f34e35ef9d6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.582636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.583508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.584199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.584199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.586929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.588464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.591492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 49/91] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.592359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.599771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f90ac-c392-4fd7-9260-856d9aaa9120 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.639008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:23.651847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.652830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.653303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.653755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.654992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=beafb93941304589a5106acd52069473, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:23.655505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.657153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.663595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 43/92] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=beafb93941304589a5106acd52069473&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.664436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.832353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:23.840112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfa6f4-6633-42a9-a3c7-011f331ad7ee None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:23.907233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:23.910287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:23.912757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:23.915314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:23.918431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:23.922109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:23.927328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 50/93] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:23.932916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:23.957817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd0c4-cdb8-471b-9a5d-6fa6fd752bcf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:24.025085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:24.025778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:24.026085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:24.026418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:24.027278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=beafb93941304589a5106acd52069473, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:24.027602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:24.029076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:24.042220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 44/94] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=beafb93941304589a5106acd52069473&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 259 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:24.116156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:24.126141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285405-2c5b-4725-95a1-f96b2bc4f54b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:24.174845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:24.175530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:24.175826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:24.176113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:24.282041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:24.283517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:24.422928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:24.431105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 51/95] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:24.437270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:24.595295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee798c94-1f74-4a77-b62a-ee6ccfb6e575 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:24.597642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:24.634150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:24.635323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:24.635814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:24.636270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] PATH_INFO: `/v3/projects/4dead3c476ca48beb8c40840b2ff4231` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:24.637678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] RBAC: Authorizing `identity:get_project(project_id=4dead3c476ca48beb8c40840b2ff4231)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:24.639477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fac541-1c41-4e7a-813d-4d41153f4197 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:24.641380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 45/96] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:24 2022] GET /identity/v3/projects/4dead3c476ca48beb8c40840b2ff4231 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:24.649809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:24.814832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555987d8-e1c3-4657-8fba-2ad3d4b18b21 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:24.865849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:24.867069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:24.867359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:24.867642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:24.868487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=4dead3c476ca48beb8c40840b2ff4231, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:24.868809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:24.870315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133de603-3b91-4849-89ee-db37bae24169 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:24.881932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 52/97] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:24 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=4dead3c476ca48beb8c40840b2ff4231&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:25.903579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bb7f-55eb-445d-8bff-775ee3ab2995 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:25.904730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bb7f-55eb-445d-8bff-775ee3ab2995 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:25.905288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bb7f-55eb-445d-8bff-775ee3ab2995 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:25.909341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 46/98] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:25.916037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:25.916963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:25.917447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:25.941967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:25.943529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:26.028728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:26.046378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 53/99] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:26.086639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40bb7f-55eb-445d-8bff-775ee3ab2995 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:26.235209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40bb7f-55eb-445d-8bff-775ee3ab2995 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:26.303066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:26.303903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.304191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.304471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.322698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0ac19d4c-1520-4750-a174-fb354825bdb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:26.330926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:26.332254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:26.334153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 47/100] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:26.339260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:26.384875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc57b-ef8d-4f49-9b7d-ca161081f021 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.386385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc57b-ef8d-4f49-9b7d-ca161081f021 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.387073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc57b-ef8d-4f49-9b7d-ca161081f021 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.389889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 48/101] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:26.400138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48820-5191-4d69-b926-2c3de3b7efc2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.400749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48820-5191-4d69-b926-2c3de3b7efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.401235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48820-5191-4d69-b926-2c3de3b7efc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.428836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f1a48820-5191-4d69-b926-2c3de3b7efc2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:26.435979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f1a48820-5191-4d69-b926-2c3de3b7efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:26.483906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:26.549547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 49/102] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:26.583088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479b82-e909-4bff-9d70-a5f8af7ac985 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.583393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479b82-e909-4bff-9d70-a5f8af7ac985 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.583832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479b82-e909-4bff-9d70-a5f8af7ac985 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.586203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 50/103] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:26.588230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad96b4-00cf-4536-a3e1-911be3cd5b99 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:26.594344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45479b82-e909-4bff-9d70-a5f8af7ac985 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:26.653302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:26.654426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.656139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.656660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.657676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:26.659016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:26.661839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596235-61d9-4e36-afdc-f920b184a200 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:26.684597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 54/104] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:26.686555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.687090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.687597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.717917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:26.720693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:26.787790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45479b82-e909-4bff-9d70-a5f8af7ac985 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:26.824560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:26.842747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:26.844506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.845030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.845540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.847137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:26.847676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 55/105] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:26.848580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:26.849486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64861a88-8d44-47a4-9d22-9e21c4967cf5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:26.851609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 51/106] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:26.853192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354c71-1c95-46ad-830c-ef79b9ce4bc7 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:26.853705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354c71-1c95-46ad-830c-ef79b9ce4bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:26.854231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354c71-1c95-46ad-830c-ef79b9ce4bc7 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:26.856061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 52/107] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:26.857365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03354c71-1c95-46ad-830c-ef79b9ce4bc7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:26.906421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.001072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf26fc-2bff-4406-a515-b192a02e11c0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.021126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03354c71-1c95-46ad-830c-ef79b9ce4bc7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.049735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.051569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.052556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.053304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.055036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d127b-263a-4025-9a3a-c2a18b4aa0a3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.059334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 56/108] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:26 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:27.061037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.061496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.062624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.068047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.069370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.069738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.070120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.071376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.073020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.086212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:27.089032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:27.149181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.164215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 53/109] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:08:26 2022] POST /identity/v3/projects => generated 302 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:27.166821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.191653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 57/110] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:27.193394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.298025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.326097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e021-9f6c-4cd5-acf9-6d3236dd2a55 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.350354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a5ec6-fac2-4b77-be5b-8248072e40e3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.374903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.376175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.376749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.377044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.392841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:27.397110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.398619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.398619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.398619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.399765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.401214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55990f20-f291-487d-8c85-146043e831d6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:27.401309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:27.402141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:27.402916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:27.403508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 58/111] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:27.403573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 54/112] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:27.404318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b859-1aed-433e-b784-2c6fa18c7da4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.404429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.404778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b859-1aed-433e-b784-2c6fa18c7da4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.405065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b859-1aed-433e-b784-2c6fa18c7da4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.406557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 59/113] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:27.411227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534b859-1aed-433e-b784-2c6fa18c7da4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.480552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.536383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c420d-af40-44f3-a8f8-88bc603ebdbb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.539180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534b859-1aed-433e-b784-2c6fa18c7da4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.575117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.575926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.576217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.576500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.586491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.589569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:27.590131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.591058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.591549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.592031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.593036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:27.593518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.595135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5dcf-3224-416a-89bf-07dc4fa02c42 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:27.598183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:27.605923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:27.606941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 55/114] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:27.607672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 60/115] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:27.607750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.607816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.607816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.607816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.629830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:27.631571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:27.747253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 61/116] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:27.748125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.750462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aca985-abd6-40de-9c14-987c804f1db9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.801255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.802202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.802499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.802792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/users/2be999d55fd24e54ae049877c17b8c43/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.803083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:27.805414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, user_id=2be999d55fd24e54ae049877c17b8c43, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.806930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.868593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea239f-041b-4a58-b49c-db13bb9ca28e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:27.872315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 56/117] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:27 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/users/2be999d55fd24e54ae049877c17b8c43/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:27.874335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.944299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:27.945179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:27.946444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:27.946755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:27.947698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:27.948027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:27.949799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:27.962862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 62/118] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:27.966288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:27.984424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.035289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3886-d50c-4ad6-8b77-718226c00101 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.080433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.081335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.081628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.082040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.083300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.084777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59fe2-83f7-4b0f-a880-1ec208838e3a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.087360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 57/119] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:28.088684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579679-95f2-40dd-9a9a-1d385f6958c7 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.088984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579679-95f2-40dd-9a9a-1d385f6958c7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.089275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579679-95f2-40dd-9a9a-1d385f6958c7 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.090798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 58/120] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:28.091526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41579679-95f2-40dd-9a9a-1d385f6958c7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.115233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00baf-26f5-41c1-9d51-033a8362793b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.117661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.162525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.164012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.164304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.164594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.178720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3cccec60-1c61-4763-b974-ff54f77c5872 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:28.192775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:28.200193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 63/121] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:28.201146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.202162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.202162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.208850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41579679-95f2-40dd-9a9a-1d385f6958c7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.220534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:28.222284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:28.246573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.254609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.255837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.261567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.261567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] PATH_INFO: `/v3/users/303d7a55c3264706890a468f4b94939a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.271522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] RBAC: Authorizing `identity:get_user(user_id=303d7a55c3264706890a468f4b94939a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.273748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.275859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 59/122] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/users/303d7a55c3264706890a468f4b94939a => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:28.276667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.318330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 64/123] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:28.319249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.400581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bc05e-50c8-4cf4-a8c7-277502eb93c8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.440640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.441329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.441623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.442050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.443384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.445580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.447834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 60/124] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:28.448518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.450592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.463366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0a7a8-8d4b-4950-8d38-4548c1c342a4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.506492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.507238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.507533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.507821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.508651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:28.508975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.510462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.520089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 65/125] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:28.521069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.521507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.580460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb478c5e-a22f-4d12-a564-b9bcc25a035f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.634418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.635145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.635446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.635738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.649460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28432ef7-dfae-4756-9b06-d41004ac5628 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.698767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.699502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.699796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.700076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.717533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:28.723945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.729796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:28.729978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:28.732014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 66/126] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:28.734319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.757940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.789133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:28.839839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 61/127] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:28.840719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:28.866464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc062a-b98b-4d7e-8951-a7106b648c38 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.904297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.905441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.906001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.906397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/users/64e0572a2aa94e349233dd92e98da7b7/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.909156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, user_id=64e0572a2aa94e349233dd92e98da7b7, role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:28.910687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:28.959909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82062302-8eb0-4106-b0fe-b8b7f7c4dd4b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:28.996826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:28.997713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:28.998036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:28.998320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:28.999557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.001046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.002732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 62/128] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:28 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.003528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.012985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 67/129] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:28 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/users/64e0572a2aa94e349233dd92e98da7b7/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:29.014989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.047024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.146320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.186023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890151c-36ac-4e06-a3e5-32be61867d4c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.236164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663f28-d14f-4f88-a425-2325a30e5de9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.244595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.245913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.246404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.246877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.248384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.250771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.252868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 68/130] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.253907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.309093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.309946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.310264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.310498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.311290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:29.311664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.313080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.324041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 63/131] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.325282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.398759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec00fb2-62e0-4477-bccb-399711a1a7f8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.435234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.436200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.436495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.436843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.438582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.440741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.442395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 69/132] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.443026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.455576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.457383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.475498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a6c9-79c2-4f6f-bdc9-5224bc6d8cad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.525733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.527276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.527894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.528516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.529845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, scope.project.id=beafb93941304589a5106acd52069473, user.id=303d7a55c3264706890a468f4b94939a) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:29.530419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.532131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.539217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 64/133] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/role_assignments?user.id=303d7a55c3264706890a468f4b94939a&scope.project.id=beafb93941304589a5106acd52069473&role.id=566fbb0f211942929d530812c959c53a => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.540450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.570114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a77df-43e3-4254-bc4f-3dcdd834afed None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.612731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.614036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.614036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.614240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.615635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.617121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae575d75-00c3-4938-b9f4-92679ce8e4c0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.619007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 70/134] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.620295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f5ea1-3317-4048-9ae1-02d248a807b2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.620601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f5ea1-3317-4048-9ae1-02d248a807b2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.620890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f5ea1-3317-4048-9ae1-02d248a807b2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.622253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 71/135] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:29.626772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f5ea1-3317-4048-9ae1-02d248a807b2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.663422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52d24-7ee3-469e-9c4d-d690a5da6d41 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.703616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.704898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.705378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.706130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/users/64e0572a2aa94e349233dd92e98da7b7/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.708634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, user_id=64e0572a2aa94e349233dd92e98da7b7, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.710986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884c59-57d4-437a-a92a-3dae7807555d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.718025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.749551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f5ea1-3317-4048-9ae1-02d248a807b2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:29.781563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 65/136] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:29 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/users/64e0572a2aa94e349233dd92e98da7b7/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:29.783388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.784011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.784552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.816978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:29.818762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:29.821786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:29.831480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:29.832544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:29.832934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:29.833299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] PATH_INFO: `/v3/projects/4dead3c476ca48beb8c40840b2ff4231` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:29.836154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=4dead3c476ca48beb8c40840b2ff4231)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:29.837813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:29.839756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 72/137] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/projects/4dead3c476ca48beb8c40840b2ff4231 => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:29.847876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:29.931536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 66/138] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:29.977030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:30.055867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59e08a-348d-486f-9aee-def32aef2fb1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:30.131848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:30.132854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.133144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.133423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] PATH_INFO: `/v3/projects/4dead3c476ca48beb8c40840b2ff4231/users/64e0572a2aa94e349233dd92e98da7b7/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.143764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] RBAC: Authorizing `identity:create_grant(project_id=4dead3c476ca48beb8c40840b2ff4231, user_id=64e0572a2aa94e349233dd92e98da7b7, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:30.144626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:30.176992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea85c-9fe0-4656-888a-8f43b8fe339b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:30.213193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 73/139] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:29 2022] PUT /identity/v3/projects/4dead3c476ca48beb8c40840b2ff4231/users/64e0572a2aa94e349233dd92e98da7b7/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:08:30.247668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:30.248984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.249456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.249944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.254963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:30.258113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0996935-8d1c-4a8b-85b3-797ccbbdf641 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:30.261157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 67/140] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:30.267287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:30.283810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:30.385630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce248233-fa9e-4355-83f1-e26fac66dcce None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.386323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce248233-fa9e-4355-83f1-e26fac66dcce None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.386801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce248233-fa9e-4355-83f1-e26fac66dcce None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.388466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 68/141] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:30.394173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.394800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.395396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.411167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:30.420537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db53f3-2a34-478d-9b25-bed1555cc597 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:30.425217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:30.426707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:30.519475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 69/142] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:30.535730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:30.536437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.536724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.537011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.537835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:30.539361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:30.558035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:30.635479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 74/143] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:30 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:30.693029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bfca-2ff7-4579-9bea-c591c84552e9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:30.737308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:30.738170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:30.738465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:30.738751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:30.742206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:30.757645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:30.765739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:30.766661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:30.767416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 70/144] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:30.775141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:30.886139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:30.953406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ec5b-42b9-4ff8-9dfb-c8556a027f6f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:31.001523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:31.002247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:31.002532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:31.002809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:31.003590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:31.003898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:31.005294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:31.017066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 75/145] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:31.023356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:31.198997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ff4cb-49ea-4cdc-9fe9-b1f8a19ab784 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:31.255493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:31.257199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:31.257624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:31.258059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] PATH_INFO: `/v3/users/2be999d55fd24e54ae049877c17b8c43` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:31.259479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=2be999d55fd24e54ae049877c17b8c43)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:31.261078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:31.262892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 71/146] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:31 2022] GET /identity/v3/users/2be999d55fd24e54ae049877c17b8c43 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:31.268510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:31.417237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa20af4-82c5-48c9-be25-59d4f48a1dfa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:31.453842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:31.465703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:31.466580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:31.466924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:31.467208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:31.557500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:31.558963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:31.695663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 76/147] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:31.701729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:31.792958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:31.840034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5598-c9b5-416a-bf42-c4b2c1695ea3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:31.885192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:31.886088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:31.886544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:31.886837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:31.888051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:31.889821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d58ba4-e0a4-49fd-a78f-16e737436c59 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:31.891887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 72/148] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:31 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:31.899947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:31.975804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:32.125883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52e128-e97b-4c8a-a4a8-b4566d64a214 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:32.179800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:32.180496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.180786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.181073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.181967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=beafb93941304589a5106acd52069473, user.id=2be999d55fd24e54ae049877c17b8c43) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:32.182290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:32.183765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2cfc8-11fb-4752-b60f-264885d80acc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:32.191642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 77/149] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:31 2022] GET /identity/v3/role_assignments?user.id=2be999d55fd24e54ae049877c17b8c43&scope.project.id=beafb93941304589a5106acd52069473&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:32.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8505a-2b93-429d-a8b4-8843105200b2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.299618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8505a-2b93-429d-a8b4-8843105200b2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.299888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8505a-2b93-429d-a8b4-8843105200b2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.302580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 73/150] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:32.311307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6407-918f-4b72-aacb-03f53549eb2a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.312018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6407-918f-4b72-aacb-03f53549eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.312018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6407-918f-4b72-aacb-03f53549eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.339158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-199a6407-918f-4b72-aacb-03f53549eb2a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:32.341155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-199a6407-918f-4b72-aacb-03f53549eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:32.462765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 78/151] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:32.498895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8505a-2b93-429d-a8b4-8843105200b2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:32.575963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7cd59-7cad-4477-a8eb-d1a83581873b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.576314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7cd59-7cad-4477-a8eb-d1a83581873b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.576529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7cd59-7cad-4477-a8eb-d1a83581873b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.577955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 79/152] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:32.583254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.583544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.584371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.615989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:32.617214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:32.651477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:32.673570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8505a-2b93-429d-a8b4-8843105200b2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:32.764208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 80/153] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:32.770127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:32.783740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:32.786431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:32.786431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:32.786431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:32.801251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:32.807042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:32.821467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:32.822617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:32.823877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 74/154] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:32.830885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.022591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b8fce-03af-4c87-8a9f-87adc4d04780 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.067913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0be9-93e8-45fe-9396-0dac9e1c92b4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.107613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.108880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.109358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.109886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.135047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1e8fddae-0008-40bb-bf2a-5ffae97e916e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:33.143399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:33.144513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:33.145467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:33.147164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.147898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.148191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.148472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.149240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:33.149352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 81/155] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:33.150832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ea3e-192c-4b8c-9bfa-b2c012926b8e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.151336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ea3e-192c-4b8c-9bfa-b2c012926b8e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.151817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ea3e-192c-4b8c-9bfa-b2c012926b8e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.153282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 82/156] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:33.154082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:33.155086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74ea3e-192c-4b8c-9bfa-b2c012926b8e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.155952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e3de6-8e1a-480f-8499-133e1280e09c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:33.167922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:33.169883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 75/157] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:33.171341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.171635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.171915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.207330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:33.209954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:33.305576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:33.323750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74ea3e-192c-4b8c-9bfa-b2c012926b8e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.341704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 76/158] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:33.343086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.398967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.400064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.400559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.401042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.403171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:33.403687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:33.405724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:33.427238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 83/159] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:33.429804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.540646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a500-1596-468e-8531-088c0ffd69e9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.574268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:33.584581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:33.597950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 77/160] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:33.598456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.620466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88236b75-b26f-405a-a82c-965ee95259df None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.703169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.703978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.704261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.704658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.723895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:33.731782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:33.732659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:33.733454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 84/161] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:33.737013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:33.737681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.844230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d201b-f9ae-4ba9-a87c-f1f4df09482b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.926985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5c52b-dfec-4228-a011-c6315488f0dd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:33.927402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.928929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.929267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.929546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] PATH_INFO: `/v3/users/303d7a55c3264706890a468f4b94939a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:33.930823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] RBAC: Authorizing `identity:get_user(user_id=303d7a55c3264706890a468f4b94939a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:33.932298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:33.934174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 78/162] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/users/303d7a55c3264706890a468f4b94939a => generated 289 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:33.943627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:33.965175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:33.972222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:33.975544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:33.975544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:33.975544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.048894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.050744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.160077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 85/163] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:34.161131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.192702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:34.231950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5bb53-8037-47fe-b0ce-0b6e75e42250 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:34.269923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e110d6-6377-40cb-9d14-f6ae3742879c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:34.290320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:34.313613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:34.314751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.315109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.315457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.316823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.318958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0b747-0332-4b8f-9e29-0239f9f21625 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.320832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 86/164] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:34 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:34.322380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590cc2-7b1f-4b23-937a-7b14e5274b3b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.322674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590cc2-7b1f-4b23-937a-7b14e5274b3b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.323215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590cc2-7b1f-4b23-937a-7b14e5274b3b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.324952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 87/165] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:34.326333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590cc2-7b1f-4b23-937a-7b14e5274b3b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.342609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:34.343307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.343605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.343889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.344692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:34.345008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.346512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.364203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 79/166] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:34.365600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.437153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:34.442661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590cc2-7b1f-4b23-937a-7b14e5274b3b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:34.481128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:34.482036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.482325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.482604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.483823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.485278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a469d-614a-43d8-b6f1-2de16a90adb0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.487188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 88/167] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:34 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:34.488530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6732-4536-409e-80ae-112a24977d7b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.488823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6732-4536-409e-80ae-112a24977d7b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.489106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6732-4536-409e-80ae-112a24977d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.507453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-263a6732-4536-409e-80ae-112a24977d7b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:34.508684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-263a6732-4536-409e-80ae-112a24977d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:34.586952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 89/168] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:34.587921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a6732-4536-409e-80ae-112a24977d7b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.672222 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412bdb3-704c-4903-aa54-1866cadde5f3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:34.706326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:34.713372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a6732-4536-409e-80ae-112a24977d7b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:34.753839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:34.754754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.755041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.755330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/users/303d7a55c3264706890a468f4b94939a/roles/566fbb0f211942929d530812c959c53a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.759570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:34.760482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:34.760769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:34.761047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:34.763217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.766275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.766596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 90/169] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:34 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:34.768754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, user_id=303d7a55c3264706890a468f4b94939a, role_id=566fbb0f211942929d530812c959c53a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:34.770545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.770959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:34.847632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 80/170] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:34 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/users/303d7a55c3264706890a468f4b94939a/roles/566fbb0f211942929d530812c959c53a => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:34.848557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:34.944038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:34.952351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc359ba3-8fee-4c13-8d73-97fb8912f4c5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.021661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1dbfd-a7a3-4fa4-a9ab-9fc3efafaf27 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.035985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.038881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.039418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.040223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.041697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=beafb93941304589a5106acd52069473, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:35.042285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:35.045432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:35.062827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 91/171] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:34 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=beafb93941304589a5106acd52069473&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:35.064088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:35.106442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.108087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.108603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.109142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.131341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:35.142166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a3de8d27-c030-4bfe-80b3-eb45eabd13a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:35.146803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:35.147813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:35.148613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:35.149218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 81/172] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:35.151447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0859115-f83b-4622-9377-206badb74c41 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.151975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0859115-f83b-4622-9377-206badb74c41 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.159076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0859115-f83b-4622-9377-206badb74c41 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.159076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 82/173] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:35.159076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0859115-f83b-4622-9377-206badb74c41 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:35.240130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30984-0f00-4cca-9e64-1372a239be1a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.313474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.314099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.314388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.314669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.420866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0859115-f83b-4622-9377-206badb74c41 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.425215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:35.458008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:35.460089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:35.502146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:35.520258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.520951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.521937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.522225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.522994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:35.523300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:35.524747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f551f8-3f56-4e57-bf5e-4a484515f08b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:35.541287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 83/174] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:35.542461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebe463-4cbb-40ef-ad0b-cdcb1cd31be6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.542759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebe463-4cbb-40ef-ad0b-cdcb1cd31be6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.543040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebe463-4cbb-40ef-ad0b-cdcb1cd31be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.574184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f0ebe463-4cbb-40ef-ad0b-cdcb1cd31be6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:35.575435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f0ebe463-4cbb-40ef-ad0b-cdcb1cd31be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:35.606201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 92/175] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:35.607924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:35.739041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 84/176] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:35.741975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbec64-0b97-4939-a94b-17f2d562b442 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.741975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbec64-0b97-4939-a94b-17f2d562b442 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.741975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbec64-0b97-4939-a94b-17f2d562b442 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 85/177] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41dc53-bad6-43ca-8b87-521202e100d1 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41dc53-bad6-43ca-8b87-521202e100d1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41dc53-bad6-43ca-8b87-521202e100d1 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 86/178] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:35.747184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41dc53-bad6-43ca-8b87-521202e100d1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:35.785126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe138c55-0557-4bb6-8786-7d015f5fe0d3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.823319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:35.835163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.835458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.835663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.835941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] PATH_INFO: `/v3/users/64e0572a2aa94e349233dd92e98da7b7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.837711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=64e0572a2aa94e349233dd92e98da7b7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:35.839661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218356-dc69-4961-bf39-1f03e4219bc7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:35.846400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 93/179] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:35 2022] GET /identity/v3/users/64e0572a2aa94e349233dd92e98da7b7 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:35.846400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.846400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.846400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.858984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:35.864366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:35.865605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:35.866738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41dc53-bad6-43ca-8b87-521202e100d1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:35.910366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:35.911257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.911576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.912390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.913614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:35.915119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db73b5-e813-47a3-be96-10ba50874638 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:35.921823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 87/180] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:35 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:35.921823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:35.921823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:35.921823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:35.958529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:35.961315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:35.973803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 94/181] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:35.973803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.071862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 88/182] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:36.073347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.113684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebef2eb-54fb-4244-995f-2be302f81698 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.151877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.163308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.170580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.170580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.170580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.176042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:36.179000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.187405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b034b6-b278-44fa-9d37-eac11713e927 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:36.192729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:36.193653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:36.194519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:36.195144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 95/183] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:36.199066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.228123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.228874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.229245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.229587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.311607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.313368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.327070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccbb8a-3783-48fe-87ec-1d69fe04722b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.366528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.367124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.367409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.367688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.368531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=beafb93941304589a5106acd52069473, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:36.368854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.370367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.377040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 96/184] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=beafb93941304589a5106acd52069473&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:36.377040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.427158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.440353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 89/185] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:36.441538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.450862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.511788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d647-02c2-4c29-85af-7a36b3817c21 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.552912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.553552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.553868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.554221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.555391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.556938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.558842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 97/186] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:36.559923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.560287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7893a-4051-49f3-82e7-30b6a1dfd10f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.603778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.604610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.604895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.605178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.627940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:36.632113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:36.633240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:36.634295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:36.636202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 90/187] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:36.636660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.696563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4975-7df2-45b0-a8b2-4eb092ff127f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.705344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.718065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.737187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.737893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.738206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.738480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.739242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:36.739552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.740960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.754204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 98/188] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:36.754204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.766102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d1c3-8bab-453a-830c-bdb287b03bfe None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.807438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.808336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.808956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.808956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] PATH_INFO: `/v3/projects/4dead3c476ca48beb8c40840b2ff4231` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.810344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] RBAC: Authorizing `identity:get_project(project_id=4dead3c476ca48beb8c40840b2ff4231)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.812314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.813568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 91/189] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/projects/4dead3c476ca48beb8c40840b2ff4231 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:36.814151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:36.878408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb9f71-2744-4da2-9113-0be3c004e1ab None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.919372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.920061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.920352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.920676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.921603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.923220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.936124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d5ec-4a26-4d50-90cb-341273ab74ca None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:36.959500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:36.974633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:36.975310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:36.975597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:36.975877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:36.976633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:36.976942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:36.978471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:36.986761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 92/190] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:36.987588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.034445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 99/191] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:36 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:37.035233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.086124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.123579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb168-fa08-4f06-9193-366bf4c81fa9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.166943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.168210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.168696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.169159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.174309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7e651-aaee-409a-9de7-2868121f2a69 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.186893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:37.188734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:37.191047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 93/192] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:36 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:37.192091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.219709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.220501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.220865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.221228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.222210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=4dead3c476ca48beb8c40840b2ff4231, user.id=64e0572a2aa94e349233dd92e98da7b7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:37.222536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:37.224042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:37.231772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 100/193] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/role_assignments?user.id=64e0572a2aa94e349233dd92e98da7b7&scope.project.id=4dead3c476ca48beb8c40840b2ff4231&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:37.232635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.250030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.326105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49697-2188-4a4b-9b06-a89c5c0adacf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.383925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.385975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.387195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.387710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.387892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70864-216d-4097-bafb-04230f3b7d4d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.388364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.390062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:37.391957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb1c8-2d78-4f18-bf9c-096502d50867 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:37.394229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 94/194] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:37.395593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c55eb-4501-4285-a1b7-336d878b6723 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.401883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c55eb-4501-4285-a1b7-336d878b6723 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.401883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c55eb-4501-4285-a1b7-336d878b6723 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.401883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 95/195] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:37.403596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.403596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.403879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.430850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:37.433305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:37.439881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.440549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.440832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.441106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.561902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:37.563947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 96/196] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:37.565007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.565115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:37.636290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.721196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.725261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 101/197] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:37.729798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:37.737936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80253-7728-4bdc-8a61-3c93bffa9f59 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.794667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.795543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.795837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.796208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.855234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21086401-9af9-4f5a-8ecc-417b4843de47 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:37.879257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:37.881303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:37.895725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:37.897334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:37.897805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:37.898262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:37.916825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:37.930448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:37.938328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:37.939264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:37.940041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 102/198] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:37.940690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.023672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 97/199] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:38.025424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.026226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.081551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01639c1-4e6b-496c-8ec1-5aaccc3a7ed5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.135197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.136598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.137008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.137482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.139622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.140272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519cbc3-54fb-45a2-a6f4-e5d720059183 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.141512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.144078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 103/200] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:37 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.144493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.179232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.180292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.180790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.181263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.182254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:38.182761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.184487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.191030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.195599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 98/201] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.201318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.267671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae008682-d101-4d7a-9354-8d6d5b2d5007 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.299929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.310009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.310907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.311189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.311467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.312732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.317820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9c03b-f20d-4e9c-807c-d00f4096f85f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.317948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.317948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 104/202] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.317948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.353889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.354752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.355134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.355587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.356571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=8206c0f1ab55481689e59cdba48d54c3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:38.356979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.358588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.365041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 99/203] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/role_assignments?user.id=8206c0f1ab55481689e59cdba48d54c3&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.366177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.438709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.440539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80f04-aa34-4cd7-86bf-48de7b94b09d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.489460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.490407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.490759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.491043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] PATH_INFO: `/v3/users/303d7a55c3264706890a468f4b94939a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.495631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185d8a-2bbb-42a5-92b7-0e5a58c2f019 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.497936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] RBAC: Authorizing `identity:get_user(user_id=303d7a55c3264706890a468f4b94939a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.499730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.501817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 105/204] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/users/303d7a55c3264706890a468f4b94939a => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.516705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:38.546367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.547267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.548531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.548814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.550682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=8206c0f1ab55481689e59cdba48d54c3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:38.550941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.558141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.569928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 100/205] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/role_assignments?user.id=8206c0f1ab55481689e59cdba48d54c3&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:38.614255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.671294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1daf5d-eaf8-4570-80cb-8b3afa7513be None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:38.713155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:38.714255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:38.714730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:38.715103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:38.820896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:38.823019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:38.914515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:38.963225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 106/206] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:08:38.968635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:39.075424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bec89c-e2a3-40ed-921f-7d407489d8e5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:39.103500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:39.111315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:39.112312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.112598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.112883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.114186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:39.118547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f587-93d9-4c73-9f2d-ff0d5d3ca047 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:39.118547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 101/207] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:38 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:39.123856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:39.276808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a7d3-776e-4469-8d2d-ce5886946860 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:39.326568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:39.327155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.327441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.327714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.328560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, scope.project.id=beafb93941304589a5106acd52069473, user.id=303d7a55c3264706890a468f4b94939a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:39.328872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:39.330392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bcb34-1a33-41a1-8db9-805c7bee6dab None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:39.337991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 107/208] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:39 2022] GET /identity/v3/role_assignments?user.id=303d7a55c3264706890a468f4b94939a&scope.project.id=beafb93941304589a5106acd52069473&role.id=566fbb0f211942929d530812c959c53a => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:39.813087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde853-538b-444d-865f-e375d78ae15c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.813600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde853-538b-444d-865f-e375d78ae15c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.813980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde853-538b-444d-865f-e375d78ae15c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.816290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 102/209] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:39.823101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.824276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.824931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.849916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:39.851545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:39.882740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:39.892505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a5e00-767f-4627-9c0f-77c2d0c10198 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.893123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a5e00-767f-4627-9c0f-77c2d0c10198 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.893630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a5e00-767f-4627-9c0f-77c2d0c10198 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.895297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 103/210] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:39.901821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:39.902215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:39.902679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:39.923808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:39.925384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:39.944095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 108/211] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:39.968960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.004268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 104/212] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:40.033820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.100276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e650e1-6c35-479a-8c99-1704e3bc74de None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.136279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.137121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.137404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.137684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.139079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:40.140621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:40.142310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 109/213] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:40.149801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.150668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0266e-4d91-42c3-99f3-4116abd04a3d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.178961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:40.184092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:40.194063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.194943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.195239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.195519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.207550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:40.221801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:40.222744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:40.223550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 105/214] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:40.225552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.263321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f733-f9f9-4f51-84a6-b5e5ef5c605a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.303165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.303922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.304213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.304602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.305496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:40.307011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:40.340482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707321e-2bf0-4db8-92a7-f7cf0da0e269 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.378000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.378705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.379036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.379319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.380647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:40.380960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:40.382515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:40.391537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 106/215] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:40.398044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.439286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 110/216] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:40 2022] POST /identity/v3/users => generated 297 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:08:40.469853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:40.542864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40739-b17d-4aab-8604-c7c01e143587 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.586695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.587667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.588003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.588291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] PATH_INFO: `/v3/users/885ccf9007f5434a84ffb755aa372a5b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.607795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] RBAC: Authorizing `identity:get_user(user_id=885ccf9007f5434a84ffb755aa372a5b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:40.612531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbe163-cfb9-4b33-b528-8b8c8ba340ab None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:40.615009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 107/217] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:40 2022] GET /identity/v3/users/885ccf9007f5434a84ffb755aa372a5b => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:40.623929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:40.722241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:40.743132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e88a97-49a4-4ad7-9b90-8cb9baae49fa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:40.783619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:40.784301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.784908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.785431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.894024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:40.895854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:40.912190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd1c8c-320b-4025-9788-c1a95a829252 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.912702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd1c8c-320b-4025-9788-c1a95a829252 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.912775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd1c8c-320b-4025-9788-c1a95a829252 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.916018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 108/218] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:40.922325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:40.923054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:40.923563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:40.963142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:40.964383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:41.024915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.055844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 111/219] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:41.069818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.106361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 109/220] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:41.129111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.206184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.247470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee953dd-db4e-4a16-aee8-975c04f1f644 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.248657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392778-3ef9-4fab-9ac2-fb11fcbf9b12 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.285702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.287302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.287588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.287872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.288533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.289093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:41.289658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.290122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.290572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.290680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f5e9-5851-4b97-8368-1fa42cff083d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:41.292625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 112/221] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:41.294142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512a47-620d-41d6-aebe-dfa0dd6f6acb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.294433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512a47-620d-41d6-aebe-dfa0dd6f6acb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.294716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512a47-620d-41d6-aebe-dfa0dd6f6acb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.296107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 113/222] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:41.300738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12512a47-620d-41d6-aebe-dfa0dd6f6acb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.304210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fad036bc-05aa-462e-842e-3379e1f997f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:41.312569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:41.313693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:41.314594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:41.316747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 110/223] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:41.318465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.318968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.319539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.339446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:41.341031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:41.357506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.413984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12512a47-620d-41d6-aebe-dfa0dd6f6acb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.420263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 111/224] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:41.421384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.466630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.494819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.495942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.496370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.496825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.498870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=885ccf9007f5434a84ffb755aa372a5b) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:41.499784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:41.503451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:41.517814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 114/225] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/role_assignments?user.id=885ccf9007f5434a84ffb755aa372a5b&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:41.518660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.558945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9d967-9099-4bf4-a7ea-3e74bcb5e4ac None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.601086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.602151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.602811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.603294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.604285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:41.604793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:41.607854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:41.619369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 112/226] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:41.624924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.692930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02268415-36b0-4d61-a4d3-bf7af0a86315 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.746144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.778098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.779338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.780024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.780896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.784257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:41.801794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:41.812304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:41.813194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:41.814001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 115/227] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:41.814659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eca348-f202-4103-bdd2-4429dc5b26c5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:41.819957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.893978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:41.894906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:41.895190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:41.896026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:41.901199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:41.902685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:41.906039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 113/228] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:41.914820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:41.980158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c76b2-833d-4948-bc96-46f1782bddab None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.018018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.018722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.019011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.019295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.020077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:42.020935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.025063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.050981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 116/229] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:42.057253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:42.099879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40cf9-3f40-4c76-85be-9d001c42040e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.135351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:42.175958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.176854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.177416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.179030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.180836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.182611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 114/230] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:41 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:42.188952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:42.218209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:42.284181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4615-8142-4a9d-9d84-d54ee0d05b0a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.295757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b7111-6527-4844-92e8-98bf37f30350 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.335586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.336485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.336768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.337097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/8206c0f1ab55481689e59cdba48d54c3/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.341439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, user_id=8206c0f1ab55481689e59cdba48d54c3, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.342931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.354253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.355156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.355549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.355827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.357048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.358639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.360287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 117/231] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:42 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:42.366206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:42.432228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 115/232] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:42 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/8206c0f1ab55481689e59cdba48d54c3/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:42.486727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb39b4f-2b86-42ee-822b-aeaf84d0a86b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.490373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:42.525799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.526754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.527269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.527555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.528870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.530557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55806-71ca-4c1a-abdc-20df6a536a0c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.532608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 118/233] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:42 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:42.539153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:42.660764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:42.685467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b78d-7b36-4c04-9a8c-f384cbe834ba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:42.727115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:42.728033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.728326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.728676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/8206c0f1ab55481689e59cdba48d54c3/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.731484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, user_id=8206c0f1ab55481689e59cdba48d54c3, role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:42.733034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087458f-5180-4932-9a2c-8fad3b02132a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:42.831477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 116/234] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:42 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/8206c0f1ab55481689e59cdba48d54c3/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:42.992662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161f542-bff0-4224-97e3-0824ac2a910d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:42.993285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161f542-bff0-4224-97e3-0824ac2a910d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:42.995124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161f542-bff0-4224-97e3-0824ac2a910d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:42.999013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 119/235] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:43.005110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.005632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.005969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.037914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:43.039915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:43.087782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7d64e-fd1b-4124-9ec1-8bcff017196d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.088099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7d64e-fd1b-4124-9ec1-8bcff017196d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.088356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7d64e-fd1b-4124-9ec1-8bcff017196d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.090084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 120/236] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:43.105847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.106204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.106487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.128623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:43.130425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:43.163704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 117/237] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:43.187335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.252248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 121/238] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 14:08:43.269610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:43.290198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.296417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978e2-e6c2-4e44-887b-60e018271053 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.333809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.334627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.334913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.335193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.351252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:43.359833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:43.360812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:43.361582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 118/239] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:43.375886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.507773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba9fd-3ce0-4504-83fd-da5aad36821c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.510071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:43.516919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81389bd-ec0f-4f3d-9e21-b6a83d02268b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.555884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.556630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.556921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.557204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:43.558400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:43.559808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:43.561506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:43.570406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 119/240] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:43.582337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.606597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.607418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.607703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.607982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.610696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:43.612234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:43.614202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 122/241] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:43.620463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.703229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637241df-03e1-45cc-ae8b-e49dd6bb85a8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.760315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.761119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.761410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.761852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] PATH_INFO: `/v3/users/e90598a4430b47f3b2fc5b6dabe67711` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.777210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] RBAC: Authorizing `identity:get_user(user_id=e90598a4430b47f3b2fc5b6dabe67711)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:43.778858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:43.780955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 120/242] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:43 2022] GET /identity/v3/users/e90598a4430b47f3b2fc5b6dabe67711 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:43.787082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:43.803268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee692f-a78e-44db-8319-541604d6bcd4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.863778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.865017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.865305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.865583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:43.866342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:43.867808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:43.876399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:43.932361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746119-6930-4bfa-b8a1-48cf2603607e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:43.974989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 123/243] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:43 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:43.976190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:43.976914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:43.977255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:43.977546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.072400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:44.075456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b488b-ed36-49b1-8e8b-35f604f6a1e9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:44.143426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:44.197937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 121/244] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:44.199740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.216789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fd02-97ec-4a09-9c69-c8971f374c1e None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.217157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fd02-97ec-4a09-9c69-c8971f374c1e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.217866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fd02-97ec-4a09-9c69-c8971f374c1e None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.219935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 122/245] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:44.225683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.225998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.226277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.234110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:44.261208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:44.262827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:44.343469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 123/246] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:44.345655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7db80-cf8c-4fcd-8adf-8b5f5f420f8d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:44.368051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.385641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:44.387072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.387620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.388047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.390198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:44.391680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:44.393618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 124/247] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:44.400206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.488050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:44.509138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d57fed-c1f9-481c-9a32-8e11c3acd0b6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:44.537584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:44.548901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32177f8e-e063-4ce1-9d6e-55ae2a2ea022 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:44.559605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:44.560735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.561261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.561853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.578924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:44.590798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:44.591732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:44.592528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:44.595942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 124/248] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:44.602191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:44.602407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.603169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.603477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.603742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.610266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=e90598a4430b47f3b2fc5b6dabe67711) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:44.610266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:44.610266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:44.615949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 125/249] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/role_assignments?user.id=e90598a4430b47f3b2fc5b6dabe67711&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=566fbb0f211942929d530812c959c53a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:44.720912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78d6-70a3-4967-a24b-2ba74da7f29f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:44.760820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:44.762017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.762498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.762969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.763959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:44.764457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:44.766097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:44.776704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 125/250] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:44.782693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.900880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac909d-1940-45a4-b6ed-5ca7f08953f3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:44.944604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:44.945492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:44.945893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:44.946281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] PATH_INFO: `/v3/users/885ccf9007f5434a84ffb755aa372a5b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:44.948324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] RBAC: Authorizing `identity:get_user(user_id=885ccf9007f5434a84ffb755aa372a5b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:44.949794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:44.951499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 126/251] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/users/885ccf9007f5434a84ffb755aa372a5b => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:44.957060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:44.977277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:45.097539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8e069-9c9e-424b-9bc1-0777444aa479 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:45.141889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:45.142767 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.143052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.143333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.144540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:45.146074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d1174-ccd7-44b6-b4c5-7c3d2a2bf6e2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:45.148087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 126/252] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:44 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:45.153595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:45.202164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:45.309496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670145-9641-4c57-89c0-3fbec408df3d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:45.353480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:45.354400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.354683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.354960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/885ccf9007f5434a84ffb755aa372a5b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.356969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, user_id=885ccf9007f5434a84ffb755aa372a5b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:45.358459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67c723-104e-4a30-9a2a-63ed65aefd15 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:45.432022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 127/253] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:45 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/885ccf9007f5434a84ffb755aa372a5b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:45.442152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f38f0-1357-4af1-aea1-3c03a5ec6f9b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.442478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f38f0-1357-4af1-aea1-3c03a5ec6f9b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.442711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f38f0-1357-4af1-aea1-3c03a5ec6f9b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.444353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 127/254] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:45.450562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff24-1937-459f-be82-25379f0d40bb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.450869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff24-1937-459f-be82-25379f0d40bb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.451113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff24-1937-459f-be82-25379f0d40bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.474710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-593fff24-1937-459f-be82-25379f0d40bb None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:45.475969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-593fff24-1937-459f-be82-25379f0d40bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:45.574652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 128/255] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:45.618285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f38f0-1357-4af1-aea1-3c03a5ec6f9b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:45.721426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:45.735907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad56306-0667-40ae-91f3-1c74a427c9ce None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.736573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad56306-0667-40ae-91f3-1c74a427c9ce None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.737059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad56306-0667-40ae-91f3-1c74a427c9ce None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.739023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 129/256] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:45.744247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.744774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.745276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.756040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f38f0-1357-4af1-aea1-3c03a5ec6f9b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:45.767818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:45.769370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:45.791444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:45.802459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:45.803258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:45.803548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:45.803996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:45.821264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:45.831127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:45.832043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:08:45.833449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:45.835398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 128/257] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:45.841325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:45.869299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 130/258] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:45.896183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.018634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540eb45-7695-46eb-9993-04c532a8be7f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.019692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddfa1f-c790-4dca-91f6-34d6e3b09813 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.063087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.064177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.064663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.065136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.089816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:46.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:46.100102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:46.101027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:46.103633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 131/259] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:46.109657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.115970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.116666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.116950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.117230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.118023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:46.118333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.119720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.121586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.126174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.142948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 129/260] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:46.165025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.234058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4acee-b607-40e8-aa94-f5ffac64f01a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.275352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.276373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.276842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.277302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.278299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:46.278828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.280530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.290863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 132/261] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:46.297348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.374159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.389294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9885-9fd4-4d0d-b63f-cc3f5955fcf3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.408456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71d0d-6dc8-440d-998f-95f96a16ce27 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.444805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.445840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.446203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.446558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.448108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.449685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.452079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 133/262] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:46.457859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.463031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.463916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.464202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.464483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] PATH_INFO: `/v3/users/8206c0f1ab55481689e59cdba48d54c3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.470653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] RBAC: Authorizing `identity:get_user(user_id=8206c0f1ab55481689e59cdba48d54c3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.472123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.474709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 130/263] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/users/8206c0f1ab55481689e59cdba48d54c3 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:46.490790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.571029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7256d-68dd-4298-814e-d00ae12dd943 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.594798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.607854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.608647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.609176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.609467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.614134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.662661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f720-5970-4451-bb11-010a38b4925a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.686547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.688039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.726484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.727207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.727496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.727772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.796220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 134/264] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:46.801971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.841473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.842894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.852749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.909456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37676-e025-4f9d-8064-0e3380bb0083 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:46.930377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:46.948086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:46.948965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:46.949259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:46.957337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:46.957337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:46.957337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:46.957337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 135/265] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:46.960068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:46.994261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 131/266] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:47.005034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:47.083287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9425e84-ef22-4a80-81ce-60783e8fc090 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:47.102251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:47.129848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:47.130661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.130960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.131240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.132206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=8206c0f1ab55481689e59cdba48d54c3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:47.132528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:47.134101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:47.146142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 136/267] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:46 2022] GET /identity/v3/role_assignments?user.id=8206c0f1ab55481689e59cdba48d54c3&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:47.172228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc976b9-7afb-4b86-a9d2-84350e3f9d5e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:47.229227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:47.230312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.230665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.230944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.234544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:47.236038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a139e-d107-4a21-a4b9-ed04db271ead None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:47.237867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 132/268] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:47 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:47.244436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:47.387120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826beb-bf13-4c2d-8256-ef24f29d8d8b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:47.423922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:47.424603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.424902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.425193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.426745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=8206c0f1ab55481689e59cdba48d54c3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:47.427391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:47.428857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42f90c-9599-435e-a72e-c148250dc945 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:47.436030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 137/269] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:47 2022] GET /identity/v3/role_assignments?user.id=8206c0f1ab55481689e59cdba48d54c3&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:47.503152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676ce2f-424a-4714-a27c-5480743cee80 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.503802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676ce2f-424a-4714-a27c-5480743cee80 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.504319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676ce2f-424a-4714-a27c-5480743cee80 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.505987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 133/270] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:47.511837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd28b-8ff1-4c10-939a-d49baef66350 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.512130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd28b-8ff1-4c10-939a-d49baef66350 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.512584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd28b-8ff1-4c10-939a-d49baef66350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.534154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-085fd28b-8ff1-4c10-939a-d49baef66350 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:47.536193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-085fd28b-8ff1-4c10-939a-d49baef66350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:47.569714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:47.626938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 138/271] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:47.662480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676ce2f-424a-4714-a27c-5480743cee80 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:47.706848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:47.791183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676ce2f-424a-4714-a27c-5480743cee80 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:47.827333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:47.828198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.828490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.828773 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.839959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:47.843107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0baf-7fcb-4529-a430-6fb0f99f3c58 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.843629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0baf-7fcb-4529-a430-6fb0f99f3c58 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.844151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0baf-7fcb-4529-a430-6fb0f99f3c58 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.846006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 139/272] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:47.849005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:47.849935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:47.850728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 134/273] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:47.853631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:47.854078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:47.854460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:47.854975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:47.883009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:47.884324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:47.955720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:47.964447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305d3a-1faf-46b7-befc-c5fd917cf195 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:47.978345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 140/274] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:48.001341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.002685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.003233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.003929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.004894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:48.005349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.006250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.007118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.021799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 135/275] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.023747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.126348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.134679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434d662-1dea-406a-aebc-0a4bd3363f7c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.140970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0e61-4815-4d87-8fd5-9ba82d39db29 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.180832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.182201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.182728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.183189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.183301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.183660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.183843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] PATH_INFO: `/v3/users/e90598a4430b47f3b2fc5b6dabe67711` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.184127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.185502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] RBAC: Authorizing `identity:get_user(user_id=e90598a4430b47f3b2fc5b6dabe67711)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.187128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0863f-f6b5-43b9-8e59-73708819fb75 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.188966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 136/276] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/users/e90598a4430b47f3b2fc5b6dabe67711 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.193445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19159487-e27f-4b5e-823e-f272f33b8ea6 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.193997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19159487-e27f-4b5e-823e-f272f33b8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.194286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19159487-e27f-4b5e-823e-f272f33b8ea6 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.195718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 137/277] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:48.196259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19159487-e27f-4b5e-823e-f272f33b8ea6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.200539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c7905ea3-192d-4637-b08f-38c87b6971ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:48.208543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:48.209797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 141/278] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:48.211595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.212608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.212608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.228692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.231888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:48.233077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:48.305426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 142/279] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:48.307341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.318276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19159487-e27f-4b5e-823e-f272f33b8ea6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.355383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.356241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.356526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.356802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.358020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.359474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.361161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 138/280] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.362126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.411582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.430480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f25fe7-4220-4a08-80f5-0e46e9d967b3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.462622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.468624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.469337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.469811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.470232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.471143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:48.471613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.472478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fce526-1ae1-47c6-9129-1642842513a9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.473907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.482883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 143/281] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.483873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.510027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.510822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.511112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.511394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.512575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.514048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.515685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 139/282] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.515685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.596447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e283-2d3e-4184-a28a-3d0b24e5a06f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.618991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a44120-f239-4a80-a734-77876f17ba37 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.630585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.631475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.631755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.632029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/e90598a4430b47f3b2fc5b6dabe67711/roles/566fbb0f211942929d530812c959c53a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.634073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, user_id=e90598a4430b47f3b2fc5b6dabe67711, role_id=566fbb0f211942929d530812c959c53a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.635505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acafdf-d7cc-4c90-9a1d-ed66a0ca45d9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.652824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.653687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.654005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.657686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] PATH_INFO: `/v3/users/be71f91b243c4e118a2ce415cdbfa549` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.668440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] RBAC: Authorizing `identity:get_user(user_id=be71f91b243c4e118a2ce415cdbfa549)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.669943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.671611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 140/283] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/users/be71f91b243c4e118a2ce415cdbfa549 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.672364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.689123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 144/284] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:08:48 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/users/e90598a4430b47f3b2fc5b6dabe67711/roles/566fbb0f211942929d530812c959c53a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65cbf-b739-4948-9966-1d3e0ef09e06 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65cbf-b739-4948-9966-1d3e0ef09e06 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65cbf-b739-4948-9966-1d3e0ef09e06 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 145/285] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:48.726163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65cbf-b739-4948-9966-1d3e0ef09e06 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:48.731970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:48.782605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e433-c651-4591-88cd-4a3a77ffa51b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.826983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.827950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.828233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.828518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.829354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.830884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.848146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65cbf-b739-4948-9966-1d3e0ef09e06 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:48.886111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:48.886846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.887160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.887449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.888294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, user.id=be71f91b243c4e118a2ce415cdbfa549, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:48.888608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:48.890098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93e15c-20ff-4b5c-b6eb-cc7edc02ef56 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:48.896006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 146/286] 10.222.0.93 () {56 vars in 1308 bytes} [Thu Jul 14 14:08:48 2022] GET /identity/v3/role_assignments?user.id=be71f91b243c4e118a2ce415cdbfa549&scope.system=all&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:48.897376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:48.897722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:48.898088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:48.916617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:48.917874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:48.933752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 141/287] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:08:48 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:49.004549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:49.010976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 147/288] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:49.040498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:49.139135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:49.178332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838289e6-49f8-437f-9f79-2119d3c0b034 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:49.235182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:49.235990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:49.236274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:49.236550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:49.248885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:49.257212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:49.258177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:49.259038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:49.263301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 142/289] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:49.264783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:49.381092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013ecbf-6cc3-405b-afc3-f129cda0d28a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:49.420963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:49.421651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:49.421980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:49.422264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:49.423031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:49.423341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:49.424721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:49.435277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 148/290] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:49.442571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:49.570360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224253-9489-48cf-a682-64812cf6f4f1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:49.620183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:49.625603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:49.626516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:49.626807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:49.627091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] PATH_INFO: `/v3/users/885ccf9007f5434a84ffb755aa372a5b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:49.634631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] RBAC: Authorizing `identity:get_user(user_id=885ccf9007f5434a84ffb755aa372a5b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:49.640118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:49.640118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 143/291] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:49 2022] GET /identity/v3/users/885ccf9007f5434a84ffb755aa372a5b => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:49.643713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:49.757172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:49.810362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c1f8-f1da-4d49-87ff-f654c05b0316 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:49.860508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:49.861109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:49.861424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:49.861706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:49.963901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:49.965453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:50.052690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:50.132847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 149/292] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:50.138626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:50.260229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5c2fa-d8ec-46c9-b8e3-026b957278fc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:50.295477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:50.296407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:50.296699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:50.297063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:50.298324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:50.299787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a37ed7-32fb-4f01-a5ac-9f0be2a25e3f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:50.301952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 144/293] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:50 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:50.308197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:50.485172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ddd55-27b2-4714-9fe4-b1be8a4ca79e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:50.538696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:50.539641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:50.540090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:50.540518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:50.541505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=885ccf9007f5434a84ffb755aa372a5b) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:50.542003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:50.543798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1622b9-facc-43f0-a1fb-ed9109141f27 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:50.555480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 150/294] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:50 2022] GET /identity/v3/role_assignments?user.id=885ccf9007f5434a84ffb755aa372a5b&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:51.166182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d42f4-1c4e-433d-b8be-692ce9b370e0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.166476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d42f4-1c4e-433d-b8be-692ce9b370e0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.166816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d42f4-1c4e-433d-b8be-692ce9b370e0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.168480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 145/295] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:51.174004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.174585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.175088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.195094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:51.196893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:51.221389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:51.284694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 151/296] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:51.310577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d42f4-1c4e-433d-b8be-692ce9b370e0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:51.398431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:51.436893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d42f4-1c4e-433d-b8be-692ce9b370e0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:51.485646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:51.486478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.486771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.487053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.498462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-56d7f96b-1849-4d97-afab-53e1aecc33f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:51.506734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:51.507645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:51.508440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 146/297] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:51.514138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:51.517637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5a8f-9084-4056-b73b-b72967f0f32a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.517960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5a8f-9084-4056-b73b-b72967f0f32a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.518244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5a8f-9084-4056-b73b-b72967f0f32a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.519720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 147/298] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:51.524420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.524715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.525004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.541660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:51.542984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:51.623291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 148/299] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:51.652498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:51.652885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f516d92-b634-4342-a411-13c27544ac43 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:51.700214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:51.700901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.701193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.701505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.702299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:51.702612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:51.704634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:51.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 152/300] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:51.722149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:51.724557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:51.742445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:51.771058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e446e3-3a1c-430a-a0c6-30cd3563c066 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:51.807543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:51.808431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.808722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.809004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.820161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d26609b1-b861-4670-935f-b784bd0c88cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:51.828128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:51.829027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:51.830092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 149/301] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:08:51.831282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416374b3-6b6b-41ef-b7d4-35ce6057c342 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.831282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416374b3-6b6b-41ef-b7d4-35ce6057c342 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.831282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416374b3-6b6b-41ef-b7d4-35ce6057c342 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.832396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 150/302] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:51.833473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d2bf-1d0f-45ca-94d2-d5bd3ae74094 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.833782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d2bf-1d0f-45ca-94d2-d5bd3ae74094 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.834130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d2bf-1d0f-45ca-94d2-d5bd3ae74094 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.835351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 151/303] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:51.836044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96d2bf-1d0f-45ca-94d2-d5bd3ae74094 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:51.840688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a8682-c389-4547-bf72-9b97ae5d06fe None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:51.876575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:51.877457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.877743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.878050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] PATH_INFO: `/v3/users/e90598a4430b47f3b2fc5b6dabe67711` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.879281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e90598a4430b47f3b2fc5b6dabe67711)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:51.880738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e7ef0-73a8-4a33-8601-446a6d5a02c4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:51.882558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 153/304] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:51 2022] GET /identity/v3/users/e90598a4430b47f3b2fc5b6dabe67711 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:51.883555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9567c-482a-475a-b696-93aa38813698 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.883846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9567c-482a-475a-b696-93aa38813698 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.884130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9567c-482a-475a-b696-93aa38813698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.900864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-79b9567c-482a-475a-b696-93aa38813698 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:51.902291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-79b9567c-482a-475a-b696-93aa38813698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:51.949690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96d2bf-1d0f-45ca-94d2-d5bd3ae74094 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:51.969252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:51.975652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 154/305] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:51.976948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:51.982194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:51.982194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:51.993628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:51.994851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:52.002033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.002816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.003095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.003369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.004186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:52.004492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.004842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.005880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.014330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 152/306] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.015735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.065084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 155/307] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:52.065964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.120844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560b64-ec45-4b26-96e4-104cd3258e7a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.155322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.155965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.156252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.156526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.175585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a616a0-2ff9-477e-9680-ff6e2bf7b32c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.206562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.212515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.213307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.213593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.213908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.224626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:52.226838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.228292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:52.232293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:52.233477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 156/308] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:52.234306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.239679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.325180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 153/309] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:08:52.326051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.335527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a48a7b-3402-45cd-8d4f-b60e34a9197f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.369730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.370675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.370954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.371225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] PATH_INFO: `/v3/users/1f516572e4d34bb5a7f7a653eb233f57` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.384134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] RBAC: Authorizing `identity:get_user(user_id=1f516572e4d34bb5a7f7a653eb233f57)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.386038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.387655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 157/310] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/users/1f516572e4d34bb5a7f7a653eb233f57 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.388414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.412332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.431960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc433399-eabd-450c-bf63-5228f8a353af None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.451082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.466779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.467505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.467793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.468074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.469306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.470797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.473079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 154/311] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.473746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.494823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04736080-4a9e-46c3-80a7-7db419213552 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.528634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.529303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.529587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.529897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.530648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:52.530958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.532326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.540497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 158/312] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.541287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.578826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20992842-9b1a-4845-a0c3-173a5c24dee2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.612967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.613861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.614145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.614423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.615607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.617064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.618980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 155/313] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.619538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.637086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.643324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866c954-5ebc-4e5c-9f17-9bd31f3341ea None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.666360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.676808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.677615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.677941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.678226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.679068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, user.id=1f516572e4d34bb5a7f7a653eb233f57, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:52.679387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.680995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.686327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 159/314] 10.222.0.93 () {56 vars in 1308 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/role_assignments?user.id=1f516572e4d34bb5a7f7a653eb233f57&scope.system=all&role.id=566fbb0f211942929d530812c959c53a => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.687118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.723266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3085f9-121d-4a25-b957-e711bfc4f0c9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.760548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.761226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.761519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.761896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] PATH_INFO: `/v3/groups` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.762557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.764062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.801421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9617-eb51-4966-b77a-4acfe1d5cdce None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:52.837982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 156/315] 10.222.0.93 () {60 vars in 1134 bytes} [Thu Jul 14 14:08:52 2022] POST /identity/v3/groups => generated 229 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:08:52.840037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:52.840171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.840247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:52.840247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:52.840421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] PATH_INFO: `/v3/users/be71f91b243c4e118a2ce415cdbfa549` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:52.841693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=be71f91b243c4e118a2ce415cdbfa549)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:52.843240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:52.845201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 160/316] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/users/be71f91b243c4e118a2ce415cdbfa549 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:52.851265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:52.927496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.969081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:52.981409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc63764-0de3-4a7e-8246-30a75f37b5a0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:53.005318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1216f6-ee51-482c-ae28-25b9881c67a9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:53.032058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:53.033476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:53.033871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:53.034360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] PATH_INFO: `/v3/system/users/be71f91b243c4e118a2ce415cdbfa549/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:53.035936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=be71f91b243c4e118a2ce415cdbfa549, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:53.037324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa54e3-85f9-4e1f-a012-6e6bf09a58b0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:53.045426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:53.048016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:53.048358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:53.048644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:53.052630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, scope.project.id=f3d461c5e38345bf802a554dc8287a59, user.id=e90598a4430b47f3b2fc5b6dabe67711) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:53.052630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:53.052630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897c275-aa63-4b10-a8e8-29f93fa4020e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:53.062060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 157/317] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:08:52 2022] GET /identity/v3/role_assignments?user.id=e90598a4430b47f3b2fc5b6dabe67711&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=566fbb0f211942929d530812c959c53a => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:53.120525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 161/318] 10.222.0.93 () {58 vars in 1334 bytes} [Thu Jul 14 14:08:52 2022] PUT /identity/v3/system/users/be71f91b243c4e118a2ce415cdbfa549/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:08:54.767698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075621-c658-4340-bd5b-bbe47868eb30 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:54.768231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075621-c658-4340-bd5b-bbe47868eb30 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:54.768231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075621-c658-4340-bd5b-bbe47868eb30 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:54.769878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 158/319] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:54.775325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea194f-c76b-4b7a-883c-8ba53d61831e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:54.775765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea194f-c76b-4b7a-883c-8ba53d61831e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:54.775935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea194f-c76b-4b7a-883c-8ba53d61831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:54.793968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e3ea194f-c76b-4b7a-883c-8ba53d61831e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:54.794972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3ea194f-c76b-4b7a-883c-8ba53d61831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:54.873364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 162/320] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:54.895127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58075621-c658-4340-bd5b-bbe47868eb30 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:54.930382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031df3-76d6-40f8-8983-a667d4d42316 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:54.930695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031df3-76d6-40f8-8983-a667d4d42316 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:54.930953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031df3-76d6-40f8-8983-a667d4d42316 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:54.932350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 163/321] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:54.937340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:54.937628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:54.937936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:54.954646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:54.955830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:55.005656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.006498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58075621-c658-4340-bd5b-bbe47868eb30 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.033911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 164/322] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:55.042529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.043305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.043589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.043861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.051064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.052602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.055596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:55.063425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:55.068333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:55.069133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 159/323] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:55.071748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.172221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e463-bb04-4e03-ba9b-63f6f7b3d006 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.183574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8602a-e561-49b6-95bf-6364afc94278 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.208801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.209598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.209926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.210207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.211383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.212849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f51b9e-ad73-4711-be28-12186a75e09f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.214846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 165/324] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:55.215686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fdc-e1a4-442b-a000-a78ce9546b11 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.215980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fdc-e1a4-442b-a000-a78ce9546b11 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.216257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fdc-e1a4-442b-a000-a78ce9546b11 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.217485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 166/325] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:55.220040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa67fdc-e1a4-442b-a000-a78ce9546b11 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.221414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.222228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.222521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.222801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.223554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:55.223856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.225226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8de7d-b8d5-4280-b29a-ebfa29468d8e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.233819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 160/326] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:55.235095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.235391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.235669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.253951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:55.255162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:55.270905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.310437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.321461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 161/327] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:55.322523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.323099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa67fdc-e1a4-442b-a000-a78ce9546b11 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.358490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.359166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.359452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.359743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] PATH_INFO: `/v3/groups` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.360437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.361928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.422224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74240c86-01f2-4ada-bab4-a62da01d2b5d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.439952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 167/328] 10.222.0.93 () {60 vars in 1134 bytes} [Thu Jul 14 14:08:55 2022] POST /identity/v3/groups => generated 226 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:08:55.440793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.469737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.470738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.471024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.471305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] PATH_INFO: `/v3/users/1f516572e4d34bb5a7f7a653eb233f57` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.472676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] RBAC: Authorizing `identity:get_user(user_id=1f516572e4d34bb5a7f7a653eb233f57)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.474401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.476315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 162/329] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/users/1f516572e4d34bb5a7f7a653eb233f57 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:55.483674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.590069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.619328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce0e9-6540-43a0-ba8a-80b18aa5d85d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.631948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:55.660601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adf18-11fe-48ae-b3c6-0868c26d276e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.668935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.670146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.672316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.672316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.695637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:55.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:55.705050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:55.705877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:55.706524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 168/330] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:55.714825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:55.717305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.717644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.717992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.718271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] PATH_INFO: `/v3/system/users/1f516572e4d34bb5a7f7a653eb233f57/roles/566fbb0f211942929d530812c959c53a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.721003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f516572e4d34bb5a7f7a653eb233f57, role_id=566fbb0f211942929d530812c959c53a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.724046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.823721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 163/331] 10.222.0.93 () {58 vars in 1334 bytes} [Thu Jul 14 14:08:55 2022] PUT /identity/v3/system/users/1f516572e4d34bb5a7f7a653eb233f57/roles/566fbb0f211942929d530812c959c53a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:08:55.884442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98d3f-e7d7-4336-a81f-7ac0db0214de None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:55.930149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:55.931323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:55.931929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:55.932472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:55.933534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:55.934115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:55.935779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:55.945544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 169/332] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:55.951780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:56.085830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8684c9-7f55-49c5-a5bf-648bd118597a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:56.109662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:56.125590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:56.126634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:56.126928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:56.127205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] PATH_INFO: `/v3/users/be71f91b243c4e118a2ce415cdbfa549` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:56.129561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] RBAC: Authorizing `identity:get_user(user_id=be71f91b243c4e118a2ce415cdbfa549)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:56.131111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd767-812e-4f27-9125-e47a6e914eaa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:56.132772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 164/333] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:55 2022] GET /identity/v3/users/be71f91b243c4e118a2ce415cdbfa549 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:56.138815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:56.178936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:56.278954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a6b4-6a4f-432a-aec7-9cab3f5ae2c9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:56.320005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:56.320675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:56.321225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:56.321506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:56.322405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] RBAC: Adding query filter params (role.id=1302304811c14a18bc5c38884ac0ed1c, user.id=be71f91b243c4e118a2ce415cdbfa549, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:56.322721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:56.324534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3b9c-187c-42a4-85ab-77c1fda187b0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:56.340422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 170/334] 10.222.0.93 () {56 vars in 1308 bytes} [Thu Jul 14 14:08:56 2022] GET /identity/v3/role_assignments?user.id=be71f91b243c4e118a2ce415cdbfa549&scope.system=all&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:57.939839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db610a1e-6462-493b-a22d-b5381768e963 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:57.941117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db610a1e-6462-493b-a22d-b5381768e963 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:57.941117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db610a1e-6462-493b-a22d-b5381768e963 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:57.942807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 165/335] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:57.948520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:57.949086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:57.949259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:57.969706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:57.971161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:58.055721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 171/336] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:08:58.093961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db610a1e-6462-493b-a22d-b5381768e963 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:58.235586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db610a1e-6462-493b-a22d-b5381768e963 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:58.281433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:58.282273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.282552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.282829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.299435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-41e65d96-0deb-4f02-90e2-8e6c332807cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:58.307798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:58.309364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:58.310845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 166/337] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:58.314836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:58.334850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:58.416453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db6325-cfcd-4484-8b44-e1c12e3fe269 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.416779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db6325-cfcd-4484-8b44-e1c12e3fe269 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.417006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db6325-cfcd-4484-8b44-e1c12e3fe269 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.418414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 167/338] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:08:58.427323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.427656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.427882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.450578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763efff-68b9-445d-a446-efe33fca9c7e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:58.459327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:58.460564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:58.530579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:58.531291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.531573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.531848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.532646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:58.532950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:58.535178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:58.542652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:58.549321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 172/339] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:58.556015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:58.576220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 168/340] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:58.619591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:58.714412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:58.762529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520ee8-006d-4029-8297-5e859c495d3f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:58.786005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e8a3e-d75d-4ee1-80d3-c800a3c08bfe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:58.837745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:58.837745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.837745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.837745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.867183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:58.868491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:58.869038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:58.869521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:58.895085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:58.903349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:58.904309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 14:08:58.905105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:58.911666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 169/341] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:58.918202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:58.954856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:58.961807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e726c-deb2-4555-a6e9-19823e69eb10 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.004360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.062983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.097626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 173/342] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:08:59.100930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b3a17-af21-4ff1-b332-326ae1fc3276 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.102444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7abee-29c8-4b0e-a68c-b2cf13c70e42 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.102888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7abee-29c8-4b0e-a68c-b2cf13c70e42 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.103315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7abee-29c8-4b0e-a68c-b2cf13c70e42 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.105352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 174/343] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:59.106925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf181-8aee-4b64-9192-a19157b0827d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.107354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf181-8aee-4b64-9192-a19157b0827d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.107774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf181-8aee-4b64-9192-a19157b0827d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.109722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 175/344] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:08:59.110667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf181-8aee-4b64-9192-a19157b0827d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.151055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.151737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.152282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.152573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.153805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:59.154133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:59.155796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51eb3c-7e80-41c8-aedc-ba742c217791 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.167044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 170/345] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:08:59.172162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970285a-db72-471f-bdfc-9c4a3429c94c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.172162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970285a-db72-471f-bdfc-9c4a3429c94c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.172162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970285a-db72-471f-bdfc-9c4a3429c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.192372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5970285a-db72-471f-bdfc-9c4a3429c94c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:59.193933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5970285a-db72-471f-bdfc-9c4a3429c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:59.249207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf181-8aee-4b64-9192-a19157b0827d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.287677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 171/346] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:59.289386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.289686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.290149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.300324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.301678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.302203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.302691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.304326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:59.306051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.308506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 176/347] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:59.309589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.321267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:08:59.322452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:08:59.358406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.369669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.429896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 172/348] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:08:59.431013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.453124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbac03-3f8f-4411-9dfd-d03838dc4032 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.496701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.497608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.497993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.498418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] PATH_INFO: `/v3/users/1f516572e4d34bb5a7f7a653eb233f57` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.500412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] RBAC: Authorizing `identity:get_user(user_id=1f516572e4d34bb5a7f7a653eb233f57)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:59.502081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.504406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 177/349] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/users/1f516572e4d34bb5a7f7a653eb233f57 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:59.505364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.585759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1afe4-07f7-409d-a7b0-c416779ecccf None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.630851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.631766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.632056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.632330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.646730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:59.648646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.651624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 173/350] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:59.655951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.672578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c3fb-381a-4bb7-ae96-a6324c5a59cf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.691123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.694300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.715076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.716292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.716882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.717360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.729391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:59.740009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:59.740959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:08:59.741752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:08:59.749078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 178/351] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:59.749078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.773178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011298e-c261-443d-bd29-247fc8c7ac25 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.814843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.815822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.816170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.816517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.829362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:08:59.839071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:08:59.840096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:08:59.840907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:08:59.845280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 174/352] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:08:59.846377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.872020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4ea9-d30d-4c10-bab1-3253c7a19220 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.913418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:08:59.914571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:08:59.915057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:08:59.915576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:08:59.916829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] RBAC: Adding query filter params (role.id=566fbb0f211942929d530812c959c53a, user.id=1f516572e4d34bb5a7f7a653eb233f57, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:08:59.917486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:08:59.919317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:08:59.933372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 179/353] 10.222.0.93 () {56 vars in 1308 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/role_assignments?user.id=1f516572e4d34bb5a7f7a653eb233f57&scope.system=all&role.id=566fbb0f211942929d530812c959c53a => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:08:59.934681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:08:59.952208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:08:59.968003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb14a2d-18a7-4ac6-ab40-338431dc88a7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:08:59.992611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:00.013174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:00.013931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.014228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.014510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.015373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=beafb93941304589a5106acd52069473) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:00.015692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:00.017320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:00.026816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 175/354] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=beafb93941304589a5106acd52069473&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:00.027840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:00.134497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce58af-67d0-4380-a258-af1fa547b392 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:00.219269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:00.220098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.220387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.220665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.221735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:00.222083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:00.223522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7185acd-6cdb-4eea-8090-fb6d7a9be83f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:00.240462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 180/355] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:08:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:00.241884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edec4-b60c-4386-ae33-0fb98f055052 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.242177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edec4-b60c-4386-ae33-0fb98f055052 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.242709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edec4-b60c-4386-ae33-0fb98f055052 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.244202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 181/356] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:00.258418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.258771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.258989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.264214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558b5a1-7aaf-4e73-923a-d03c5b875cf7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:00.284630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:00.285899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:00.330304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:00.332130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.332551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.332954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.335073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:00.335401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:00.336804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:00.355914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 176/357] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:00.357420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:00.379294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:00.402710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 182/358] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 14:09:00.403658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:00.434886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:00.502217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb1642-e31a-4d31-8e54-246f1c0b1cdb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:00.535974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d176dc-1992-4cad-ad09-059e5b5a6089 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:00.546391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:00.547057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.547341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.547614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.589540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:00.590479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:00.590766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:00.591312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:00.649317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:00.650866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:00.669978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:00.693968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:00.696653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:00.748773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:00.807310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 177/359] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:00.808296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:00.863713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 183/360] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:00.865333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:00.939223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d66170-2645-41df-845e-402a9d855e04 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:00.961226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.005180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.006041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.006327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.006605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.018798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e6d476cf-040a-4755-9f6e-21bf8fb0df27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:01.028323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:01.029643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 178/361] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:01.031163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6b202-858c-4aa0-8dc6-6b5942c9d69c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.031966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6b202-858c-4aa0-8dc6-6b5942c9d69c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.031966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6b202-858c-4aa0-8dc6-6b5942c9d69c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.031966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 179/362] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:01.036991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6b202-858c-4aa0-8dc6-6b5942c9d69c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.045715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5894-5757-46a2-ab94-e476be19c1c4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.081972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.097919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.104471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.104471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.104471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.104471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.104471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.106923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 184/363] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:00 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:01.107875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.189933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6b202-858c-4aa0-8dc6-6b5942c9d69c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.240244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.241232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.241784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.241982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.243175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.245171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33789c-0524-45f9-885a-439df0101aef None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.247099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 180/364] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:01.249506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01608399-05bb-48a9-b641-6e53cb73f8fa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.253428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.254563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.254563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.274740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:01.275953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:01.306272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.307629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.308129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.308635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.309630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:01.309972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.311378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.321417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 185/365] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:01.323423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.323985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.384237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 181/366] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:01.385894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.393600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.484940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88432dd4-1047-4a4b-8f5c-8e96eae65bfe None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.524063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76282d14-f11c-4084-85a8-bc0e573e49bc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.529899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.531086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.531585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.532133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.534721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.536401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.538695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 186/367] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:01.539722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.590530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.591798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.592280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.592770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.594278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.596955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.604629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 182/368] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:01.606137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.673663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.679149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1244dec-d4dd-4dc7-8380-9905524c2ed0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.718292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:01.739633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.740930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.741363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.741808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.769949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d368c2-ace1-4400-b11f-c3d317b09469 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.819221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:01.820036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:01.820574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:01.820860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:01.836335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:01.840668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:01.842391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:01.844977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:01.845912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:01.846800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:01.850460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 183/369] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:01.850460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.982941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09602c59-372c-4c6c-b025-f8138808fcd5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:01.983131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 187/370] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:01.983698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:01.985964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.012442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.028003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.029079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.029375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.029658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.030555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=f3d461c5e38345bf802a554dc8287a59) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:02.030871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.032342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.043770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 184/371] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:02.044647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.155100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b85e-ef8b-486c-ac56-9631e57d7e48 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.213644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.214373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.214699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.214982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.215910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=beafb93941304589a5106acd52069473) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:02.216232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.217064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371b4a-2514-4e0f-8b12-4f21dd3d50fc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.217716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.225591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 188/372] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:01 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=beafb93941304589a5106acd52069473&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:02.226541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.295030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.295640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.295931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.296207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.297496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:02.298089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.301368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.324684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 185/373] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:02.332205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.406508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.457337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.486772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a7689-8dc0-485e-9fb7-f8c6b6fbc048 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.548156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa8b1-8982-4b93-ab41-673935af951b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.557874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.558756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.559039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.559315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.560550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.562176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63da853-b9bd-4ee2-be7d-df0996b188d5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.564324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 189/374] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:02.566013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c8dc-c382-431a-b612-a31cf8d5d493 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.566282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c8dc-c382-431a-b612-a31cf8d5d493 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.566501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c8dc-c382-431a-b612-a31cf8d5d493 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.568061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 190/375] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:02.574215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.574501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.574776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.594851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:02.597372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:02.599185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.600084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.600553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.601006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.681335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.683557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.685970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 191/376] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:02.687723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.718163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.821994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 186/377] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:09:02.823167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.830169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:02.850560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929c5ab-e92b-46e6-ae39-6d27f58896a7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.901006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.902485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.902775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.903060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:02.904259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:02.906437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:02.908691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 192/378] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:02.911316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:02.950693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8cf09-2819-49a0-989c-1814ec419bbc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:02.988045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:02.988885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:02.989171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:02.989451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.004078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:03.011916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:03.014532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:03.016576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 187/379] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:03.021803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.036224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae2ef-678f-4360-9ce9-ac1f5c2876ed None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.063272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:03.092047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.093269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.093808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.095114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] PATH_INFO: `/v3/projects/df24a39f5f0a46c8b40d27ec05417d32` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.096940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] RBAC: Authorizing `identity:get_project(project_id=df24a39f5f0a46c8b40d27ec05417d32)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.099223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.102340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 193/380] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:02 2022] GET /identity/v3/projects/df24a39f5f0a46c8b40d27ec05417d32 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:03.104036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.146178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:03.163297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e825b8-67bc-40fb-9bcc-de9a6175b5ad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.217357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.218064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.218414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.218699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.219555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=f3d461c5e38345bf802a554dc8287a59) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:03.219869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.221345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.231855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 188/381] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:03.233811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.289118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ff8-119b-49a3-ba4d-afdec1552663 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.350427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:03.370153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.371338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.371636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.371929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.372896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:03.373217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.376299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.390633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 194/382] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:03.399159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.459469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb5a58-2e54-4b50-97e6-4189d9365ce9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.507934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.509612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.510226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.510666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] PATH_INFO: `/v3/groups/e378ac60a0824ef0b755ee415fb84459` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.516889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] RBAC: Authorizing `identity:get_group(group_id=e378ac60a0824ef0b755ee415fb84459)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.519817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.522149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 189/383] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/groups/e378ac60a0824ef0b755ee415fb84459 => generated 229 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:03.530186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.553721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:03.578230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeec441-598c-48b9-a6bf-6b71048e9223 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.628671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.629469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.629753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.630106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.634525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.636086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.637871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 195/384] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:03.644635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.681516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4bf4-4694-4d1a-b47f-eccc101d3dff None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.699732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:03.724554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.725724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.726294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.726796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.727898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] RBAC: Adding query filter params (group.id=e378ac60a0824ef0b755ee415fb84459, role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=df24a39f5f0a46c8b40d27ec05417d32) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:03.728450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.730277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.741241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 190/385] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/role_assignments?group.id=e378ac60a0824ef0b755ee415fb84459&scope.project.id=df24a39f5f0a46c8b40d27ec05417d32&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:03.788535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2dce-5bfa-4793-9265-338537ac4554 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:03.850629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:03.851533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:03.851822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:03.852253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:03.853547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:03.855108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487213e-46ab-45d3-a0ab-76d060bbdfba None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:03.857343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 196/386] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:03 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:03.863876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:03.997066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a004d-99db-4f06-bcfa-71dd01f71828 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:04.038954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:04.039848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:04.040131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:04.040411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/groups/60f0dd3316dd46609335a0fe5e603b93/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:04.042665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, group_id=60f0dd3316dd46609335a0fe5e603b93, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:04.044143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a561a7e-96db-48dd-8c2f-af0fbab8a7ac None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:04.134085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 191/387] 10.222.0.93 () {58 vars in 1442 bytes} [Thu Jul 14 14:09:03 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/groups/60f0dd3316dd46609335a0fe5e603b93/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:04.823816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d014b-7fc7-46bb-b616-f96ae81d5862 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:04.824650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d014b-7fc7-46bb-b616-f96ae81d5862 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:04.824650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d014b-7fc7-46bb-b616-f96ae81d5862 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:04.826127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 197/388] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:04.831771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6130cf-9103-4f9c-afb8-d017907de040 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:04.832100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6130cf-9103-4f9c-afb8-d017907de040 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:04.832364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6130cf-9103-4f9c-afb8-d017907de040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:04.859235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2c6130cf-9103-4f9c-afb8-d017907de040 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:04.860441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2c6130cf-9103-4f9c-afb8-d017907de040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:04.907229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:04.981631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 192/389] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:05.015992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d014b-7fc7-46bb-b616-f96ae81d5862 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.048211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:05.158610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d014b-7fc7-46bb-b616-f96ae81d5862 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.204969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:05.206425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.206717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.207000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.224230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:05.232064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:05.232962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:05.233743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 198/390] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:05.239179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.239808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc58b9-5629-41f1-a6a5-81538eab400a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.240646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc58b9-5629-41f1-a6a5-81538eab400a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.241126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc58b9-5629-41f1-a6a5-81538eab400a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.243012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 193/391] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:05.249058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.249630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.250226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.270054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:05.271719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:05.350213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:05.388580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81767-2aa1-455e-a876-b17d9859949a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.409756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 194/392] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:05.437566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:05.438288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.438573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.438853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.439612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:05.439969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:05.441402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:05.446004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.465841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 199/393] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:05.469369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.652186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:05.657644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9055b4-aba3-433a-afd6-594763e8b2cf None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.687804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a165d91-b865-43ba-a31e-c23de6be43e6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.740113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:05.740973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.741262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.741542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.765402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:05.768889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:05.769759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:05.770062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:05.770420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:05.773589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:05.775085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:05.776884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 200/394] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:05.782618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:05.786430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:05.787986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:05.789615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 195/395] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:05.796983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:05.810167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:05.958181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cda387-9a9b-47c0-a596-91163acd95b0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.963159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6cf7-50e5-414e-9baa-6b2cc5640140 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:05.999099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.000439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.000593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.001005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.001198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.001488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.001587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.001797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.003346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:06.003695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.003804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.005420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.006399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c6190-438e-412b-ae66-e0366c23d6be None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.012176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 201/396] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:06.015855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f0c2a-28a6-4660-9016-bd7b38c31ced None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.016718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f0c2a-28a6-4660-9016-bd7b38c31ced None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.017191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f0c2a-28a6-4660-9016-bd7b38c31ced None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.017673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 196/397] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:06.018799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.020103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 202/398] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:06.023490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f0c2a-28a6-4660-9016-bd7b38c31ced None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.048182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.140514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.164912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733c36d-b9ae-4b1c-bd34-825cbb1decf7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.170475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f0c2a-28a6-4660-9016-bd7b38c31ced None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.209162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.210109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.210389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.210766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/groups/60f0dd3316dd46609335a0fe5e603b93/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.212971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, group_id=60f0dd3316dd46609335a0fe5e603b93, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.214895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.215228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.216303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.216781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.217247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.219655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.221586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef6025-8b4d-4f4c-8031-57b3ee8a4905 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.223448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 203/399] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:06.224704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.225004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.225286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.246275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:06.247478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:06.284643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 197/400] 10.222.0.93 () {58 vars in 1442 bytes} [Thu Jul 14 14:09:06 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/groups/60f0dd3316dd46609335a0fe5e603b93/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:06.286178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.373360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 204/401] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:06.418833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.421688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.453971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39299f64-cacf-49d0-999d-7a58b2f03d80 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.494828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.495702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.496049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.496232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.497467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.500103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674d6f1-28b8-467c-873b-e4b07f5698e8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.502229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 198/402] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:06.503479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e94b7b-3fea-423e-9a95-daaff86cf492 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.503764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e94b7b-3fea-423e-9a95-daaff86cf492 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.504039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e94b7b-3fea-423e-9a95-daaff86cf492 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.505452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 199/403] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:06.511067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.511475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.512238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.529441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:06.530658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:06.533109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.567730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f1f14-4c51-4975-8f02-77b40c12620e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.599653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 200/404] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:06.600495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.631299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.632187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.632468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.632746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.648620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:06.662211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:06.663267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:06.664528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:06.673093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 205/405] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:06.673093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.708838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.733542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b83ab8-3d1f-4349-93a8-a504aa6e01ed None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.770739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.771546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.771829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.772110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473/groups/60f0dd3316dd46609335a0fe5e603b93/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.774389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=beafb93941304589a5106acd52069473, group_id=60f0dd3316dd46609335a0fe5e603b93, role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:06.775990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c337b1-5571-4c75-bcae-f1bbd8a77ba5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:06.792934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd1ea-f197-4256-8197-680d4b486894 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:06.826797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:06.827653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.827933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.828207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.831667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:06.839982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:06.847548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:06.854718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 206/406] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:06.855504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.880230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 201/407] 10.222.0.93 () {58 vars in 1442 bytes} [Thu Jul 14 14:09:06 2022] PUT /identity/v3/projects/beafb93941304589a5106acd52069473/groups/60f0dd3316dd46609335a0fe5e603b93/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:06.891521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725bfeb-cda9-486b-adc4-7c183b65f446 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:06.891811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725bfeb-cda9-486b-adc4-7c183b65f446 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:06.892091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725bfeb-cda9-486b-adc4-7c183b65f446 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:06.894099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 202/408] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:06.894806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725bfeb-cda9-486b-adc4-7c183b65f446 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:06.989231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ab582-8a96-4fbf-98d2-698ea299b979 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.040011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.040655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.040948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.041500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.042121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:07.042384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.043725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30c562-6383-4d0c-ad5c-e594d0edc3b0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.051180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725bfeb-cda9-486b-adc4-7c183b65f446 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.053560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 207/409] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.054831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.057619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.057619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.082311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:07.083528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:07.098207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.099115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.099866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.100218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.101418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:07.101807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.103685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.109199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.119519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 203/410] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.120416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.154561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.184190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 208/411] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:07.186039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.271178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035e8c-d7ff-4448-90c4-4bb77fc51c22 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.317032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.318165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.318898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.319251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.321018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.323583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.325417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 204/412] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.325541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7f0d5-49e6-4ee6-a322-40b7c179f6c7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.326049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.374660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.375675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.377761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.378079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] PATH_INFO: `/v3/groups/e378ac60a0824ef0b755ee415fb84459` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.380279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] RBAC: Authorizing `identity:get_group(group_id=e378ac60a0824ef0b755ee415fb84459)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.381851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.385864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 209/413] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/groups/e378ac60a0824ef0b755ee415fb84459 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.386765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.427058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.467311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3a77-14d9-4278-8634-da425a2e6840 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.467535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.506780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.507895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.508388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.508854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.520900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:07.523236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29e60-50ef-4487-9661-f1677e360d29 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:07.529161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:07.530107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:07.530889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:07.533450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 205/414] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:07.534550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.560112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.560964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.561255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.561559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.562934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.564325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.567177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 210/415] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.568041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.645359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21fa26-40dc-4e9b-ab29-2eeb9f2945d5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.674549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.677463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176aa8-d631-4eb9-bfbd-27b2bd9a4079 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.683150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.684297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.684841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.688771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] PATH_INFO: `/v3/projects/df24a39f5f0a46c8b40d27ec05417d32` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.688771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] RBAC: Authorizing `identity:get_project(project_id=df24a39f5f0a46c8b40d27ec05417d32)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.688771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.690797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 206/416] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/projects/df24a39f5f0a46c8b40d27ec05417d32 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.691822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.704872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:07.717221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.717892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.718183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.718468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.719226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:07.719541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.721265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.730215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 211/417] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:07.731234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.800598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f97b-b0c9-468c-a831-b75032c309ff None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.837064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.838259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.838728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.838933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80c11-13c3-4067-bfa1-7445ad99718f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:07.839393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59/groups/60f0dd3316dd46609335a0fe5e603b93/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.841863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d461c5e38345bf802a554dc8287a59, group_id=60f0dd3316dd46609335a0fe5e603b93, role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.843493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.874442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:07.875337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:07.875659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:07.875892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] PATH_INFO: `/v3/projects/df24a39f5f0a46c8b40d27ec05417d32/groups/e378ac60a0824ef0b755ee415fb84459/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:07.878440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=df24a39f5f0a46c8b40d27ec05417d32, group_id=e378ac60a0824ef0b755ee415fb84459, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:07.879875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:07.913612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 207/418] 10.222.0.93 () {58 vars in 1442 bytes} [Thu Jul 14 14:09:07 2022] PUT /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59/groups/60f0dd3316dd46609335a0fe5e603b93/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:07.914305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:07.957363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 212/419] 10.222.0.93 () {58 vars in 1442 bytes} [Thu Jul 14 14:09:07 2022] PUT /identity/v3/projects/df24a39f5f0a46c8b40d27ec05417d32/groups/e378ac60a0824ef0b755ee415fb84459/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:07.995819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:08.054411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d782f3-4e6e-4d35-92ff-f366d5d77cd4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:08.107099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:08.107700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.107988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.108343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.198653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:08.200367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f1ab8-8177-4654-84b6-a468f122220e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:08.263884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:08.324398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 208/420] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:08.329998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:08.416514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:08.496644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d129-3763-48d2-a6c5-180391fbb6c1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:08.550687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8bb74-054f-4956-98a0-1ff00643d550 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.551330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8bb74-054f-4956-98a0-1ff00643d550 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.551817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8bb74-054f-4956-98a0-1ff00643d550 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.553409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 209/421] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:08.554114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:08.555313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.555798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.556386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.558731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.559100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.559562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.568408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:08.575557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:08.577895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 213/422] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:08 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:08.582900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:08.588325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:08.589557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:08.708453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 210/423] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:08.724013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4582ae5-93a5-40b0-81c9-af974e3b6a34 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:08.739420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:08.745277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:08.778722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:08.779607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.779889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.780269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.781646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:08.783141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:08.792017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 214/424] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:08 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:08.796386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:08.923521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c73a-dcec-4abd-ab07-26c6a23dc382 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:08.950969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:08.967028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:08.967606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:08.967888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:08.968165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:08.969006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=beafb93941304589a5106acd52069473) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:08.969318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:08.970897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:08.977811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995146c-68e0-4ad5-b293-0d9a68da5354 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:08.978108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 215/425] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:08 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=beafb93941304589a5106acd52069473&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:09.041037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.042108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.042399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.042681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.066380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-32c7c8b6-c9d6-4aa3-bd15-66ea405d66c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:09.074675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:09.075622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:09.076410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 211/426] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:09.083597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.151460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:09.231992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8871e-fc21-4fa3-8012-44af901d2302 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:09.276462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.277150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.277434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.277706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.278495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:09.278808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:09.280184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:09.293979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 216/427] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:09.295629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938d8f-b48f-4acd-bb47-fea742d4b901 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.295922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938d8f-b48f-4acd-bb47-fea742d4b901 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.296204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938d8f-b48f-4acd-bb47-fea742d4b901 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.297606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 212/428] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:09.300394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.306184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.306770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.307268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.329334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:09.331484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:09.427220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 213/429] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:09.438280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e27b-435f-4966-aa95-269e459bfcad None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:09.449886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:09.459666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.481507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.482688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.483176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.483666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.565087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:09.567116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:09.569366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:09.600866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87501c-e6cc-4a47-abd6-f1e3c63ea9f1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:09.665058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.666494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.667164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.667797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.694251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:09.704787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:09.705754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:09.706620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:09.711251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 214/430] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:09.718121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.719746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 217/431] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:09:09.734003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.754272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:09.867951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84d33-a75c-46dd-87c3-d75041f3aa6f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:09.871708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee4c52-8ccd-407c-8cde-d2de9da52884 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:09.919244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.921006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.921387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.921892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.923031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:09.923783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:09.924557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:09.924834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:09.925082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:09.925607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:09.925955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:09.927196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:09.927578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:09.929942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 218/432] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:09.933817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:09.936042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:09.942191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 215/433] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:09.947467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.040131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.081175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084a148-e574-439b-9ad5-1e0bb2747cda None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.090217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f4ba0-95dd-4104-bc3f-abfbd7019f44 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.134480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.134480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.135333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.135564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.137019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.138496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.139284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.140434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.140563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 219/434] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:10.145675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.146077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.146887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.224855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.229402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.231236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.284387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e4c7-4e3e-4e3e-874b-4d3fb2a45203 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.334033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.334978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.335605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.336309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.338322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=1302304811c14a18bc5c38884ac0ed1c, scope.project.id=f3d461c5e38345bf802a554dc8287a59) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:10.338830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.341485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.352954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 220/435] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=1302304811c14a18bc5c38884ac0ed1c => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:10.366256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 216/436] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:10.368410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.435146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.497262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7527b-523f-4660-8587-7e422b97218c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.536459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.537330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.537614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.537921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] PATH_INFO: `/v3/projects/beafb93941304589a5106acd52069473` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.539115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] RBAC: Authorizing `identity:get_project(project_id=beafb93941304589a5106acd52069473)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.540674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ae5f-80cc-4d45-bea7-790fa7e2c238 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.542885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 221/437] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/projects/beafb93941304589a5106acd52069473 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:10.548277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.616407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e604-6f2c-4699-8b7f-f0bb7205cd6d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.616816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e604-6f2c-4699-8b7f-f0bb7205cd6d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.617223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e604-6f2c-4699-8b7f-f0bb7205cd6d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.619084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 222/438] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:10.624594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71468292-8327-4f20-971c-ef6173ae583b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.625051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71468292-8327-4f20-971c-ef6173ae583b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.625396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71468292-8327-4f20-971c-ef6173ae583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.644350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-71468292-8327-4f20-971c-ef6173ae583b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:10.645554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-71468292-8327-4f20-971c-ef6173ae583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:10.661354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.665364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ca5c-2ea6-4255-9d67-3c53f14f1ea7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.701869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.702739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.703029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.703306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.704532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.706023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c253d9-47f1-41a7-bdff-923479ecab10 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.708765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 217/439] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:10.710812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc199-6afa-4327-bd49-b93beb6151ca None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.711103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc199-6afa-4327-bd49-b93beb6151ca None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.711385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc199-6afa-4327-bd49-b93beb6151ca None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.712791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 218/440] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:10.713969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcc199-6afa-4327-bd49-b93beb6151ca None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.735058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 223/441] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:10.736243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.737063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.737348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.752479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.758637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:10.759994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:10.828152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 224/442] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:10.828491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcc199-6afa-4327-bd49-b93beb6151ca None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.829251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.873867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.874457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.874741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.875016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.875854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=beafb93941304589a5106acd52069473) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:10.876166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:10.877644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:10.883789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 219/443] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=beafb93941304589a5106acd52069473&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:10.884647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:10.930918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:10.950436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ca18e-1f48-40bf-b224-c0f584b74ded None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:10.991008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:10.991837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:10.992127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:10.992415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:10.995873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.007018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:11.015427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:11.016519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:11.017426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 225/444] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:11.024035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.026282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e92c-9b46-418e-aebc-8467e9cf7a58 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.070494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.071540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.072298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.072979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.087357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:11.095987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:11.096914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:11.097717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 220/445] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:11.103078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.154178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46fc9d-703e-4404-9124-1738eddb442f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.203441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.204129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.204418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.204701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.205491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:11.205857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.207253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.211939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.217675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 226/446] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.223332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.223476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7695d-9b72-4c78-a9b6-6c52c6255ce3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.262823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.263825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.264302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.264650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.265760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:11.265977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.267402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.277881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 221/447] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.284002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.302299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.356622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f753a0-82c9-4351-aed2-625c222ec33b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.401192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.402252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.402724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.403139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919fad6-ef5d-42e3-aff0-7bd89668b3ec None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.403598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.440749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.441323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.441613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.441921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.463016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.477003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.478607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.508654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.510269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.535515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.577233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 227/448] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:11.582781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.609273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 222/449] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 14:09:11.615430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.667286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.680395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8138-4434-4e83-a582-d9a4238c9ded None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.714110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dae18c-6db6-4b6e-94ae-ebe55e7e358b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.714708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.715573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.715863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.716154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] PATH_INFO: `/v3/projects/f3d461c5e38345bf802a554dc8287a59` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.717353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] RBAC: Authorizing `identity:get_project(project_id=f3d461c5e38345bf802a554dc8287a59)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.718833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.721293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 228/450] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/projects/f3d461c5e38345bf802a554dc8287a59 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.726477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.748087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.748945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.749234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.749510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] PATH_INFO: `/v3/projects/df24a39f5f0a46c8b40d27ec05417d32` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.750723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=df24a39f5f0a46c8b40d27ec05417d32)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.752155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.753174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.753952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 223/451] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/projects/df24a39f5f0a46c8b40d27ec05417d32 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.759170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.826697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a4614-aafd-4f8f-b0cf-e2cef8ac3e4d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.858539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175c40-e9e7-4565-8e0e-5c5962f8f9f0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:11.861098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.861988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.862281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.862559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] PATH_INFO: `/v3/groups/60f0dd3316dd46609335a0fe5e603b93` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.863750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] RBAC: Authorizing `identity:get_group(group_id=60f0dd3316dd46609335a0fe5e603b93)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.865189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.866848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 229/452] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/groups/60f0dd3316dd46609335a0fe5e603b93 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.872547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.877814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.891711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:11.893091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:11.893378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:11.893661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] PATH_INFO: `/v3/groups/e378ac60a0824ef0b755ee415fb84459` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:11.894880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] RBAC: Authorizing `identity:get_group(group_id=e378ac60a0824ef0b755ee415fb84459)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:11.896379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:11.898192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 224/453] 10.222.0.93 () {56 vars in 1180 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/groups/e378ac60a0824ef0b755ee415fb84459 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:11.903675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:11.965791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:11.973796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437de1-c603-4f79-9269-a62a241fb773 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:12.004354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909c5bc-b6f9-47f4-8606-097fc9f15178 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:12.007877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:12.008531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:12.008820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:12.009105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:12.009974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] RBAC: Adding query filter params (group.id=60f0dd3316dd46609335a0fe5e603b93, role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=f3d461c5e38345bf802a554dc8287a59) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:12.010293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:12.011733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9b5c-7137-45e3-8987-af4efda6dd6c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:12.018240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 230/454] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/role_assignments?group.id=60f0dd3316dd46609335a0fe5e603b93&scope.project.id=f3d461c5e38345bf802a554dc8287a59&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:12.038712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:12.039388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:12.039675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:12.039954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:12.040818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] RBAC: Adding query filter params (group.id=e378ac60a0824ef0b755ee415fb84459, role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=df24a39f5f0a46c8b40d27ec05417d32) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:12.041132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:12.042665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523610d-3239-466d-928c-19a5feb6bc07 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:12.049483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 225/455] 10.222.0.93 () {56 vars in 1376 bytes} [Thu Jul 14 14:09:11 2022] GET /identity/v3/role_assignments?group.id=e378ac60a0824ef0b755ee415fb84459&scope.project.id=df24a39f5f0a46c8b40d27ec05417d32&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:14.659370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b34fd-fc18-4d0d-98a9-e4bb3344566a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:14.659896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b34fd-fc18-4d0d-98a9-e4bb3344566a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:14.659896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b34fd-fc18-4d0d-98a9-e4bb3344566a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:14.661648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 231/456] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:14.667547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ba68-1516-47ab-8fb6-acb9ab55736c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:14.668278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ba68-1516-47ab-8fb6-acb9ab55736c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:14.668278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ba68-1516-47ab-8fb6-acb9ab55736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:14.691328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-78b1ba68-1516-47ab-8fb6-acb9ab55736c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:14.692522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-78b1ba68-1516-47ab-8fb6-acb9ab55736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:14.758998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:14.786249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 226/457] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:14.827298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b34fd-fc18-4d0d-98a9-e4bb3344566a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:14.872683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64510-dd24-468f-bab3-d62277ddd515 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:14.873305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64510-dd24-468f-bab3-d62277ddd515 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:14.873723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64510-dd24-468f-bab3-d62277ddd515 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:14.875539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 227/458] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:09:14.880354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:14.880640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:14.880917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:14.907780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:14.909367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:14.961329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b34fd-fc18-4d0d-98a9-e4bb3344566a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.032912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.033719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.034033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.034316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.038636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 228/459] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:15.064320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.069896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:15.103655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:15.104596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:15.105437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:15.105437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:15.105437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.105437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:15.105437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 232/460] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:15.110981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.121117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.144446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:15.236360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d062b-f1e2-43e4-a704-07b8ec831206 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.258227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:15.274013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.275191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.275626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.277904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.277904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:15.277904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.278949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f467-431e-47b9-aeca-119af2ed8905 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.287707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f4e0-6963-4eb9-8bc7-54ccbb444faf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.291723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 229/461] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:15.298064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3035-b1bd-405b-b2a7-148a56e1ccdc None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.298830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3035-b1bd-405b-b2a7-148a56e1ccdc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.299343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3035-b1bd-405b-b2a7-148a56e1ccdc None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.301553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 230/462] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:15.302941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a3035-b1bd-405b-b2a7-148a56e1ccdc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.328660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.329629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.329952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.330236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.341847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.343538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7b5a7a68-314b-4ce8-a1f0-edfa23f6b836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:15.365498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:15.366883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:15.367893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:15.367893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:15.367893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.367893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:15.375356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 233/463] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:15.377355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.377863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.378411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.407384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:15.410767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:15.419322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a3035-b1bd-405b-b2a7-148a56e1ccdc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.458974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.459644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.459931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.460202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.460914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.462853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.492439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:15.504184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 234/464] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:15.504937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.564431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 231/465] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:15 2022] POST /identity/v3/users => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:15.571190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.617557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cb004-1e64-44eb-8ea5-3fc71fffb88f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.636122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:15.661007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.661750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.662094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.662423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.663198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:15.663515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.664969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9416a9-4e66-4d2c-b8d5-c26e146691fe None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.680794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 235/466] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:15.684891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c074-9a4f-46cf-837e-73c00deab58d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.684891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c074-9a4f-46cf-837e-73c00deab58d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.684891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c074-9a4f-46cf-837e-73c00deab58d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.684891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 236/467] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:15.690442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.690761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.691046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.710312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:15.711526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:15.754996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d9774-aaf6-4107-8add-7a096608509e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:15.825427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:15.847693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 237/468] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:15.848536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:15.852560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:15.853724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:15.854230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:15.854700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:15.881203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:15.885021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:15.918864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:15.920086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:15.920935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:15.920935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:15.920935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:15.920935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:15.923859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 232/469] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:15.925105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.022730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94fb57-fd3b-4c6e-8b6d-3ca948657834 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.058483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:16.069962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.071428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.072098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.073271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.074181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.075864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.083388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603b593-2775-4f62-a691-05a057c59857 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.124533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.125347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.125637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.125945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.137828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.139292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:16.161853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:16.166036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:16.167002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:16.167002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:16.167002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:16.167002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:16.167263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 238/471] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:15 2022] POST /identity/v3/users => generated 248 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:16.167356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 233/471] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:16.167445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.168674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.315555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:16.354056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daffea-b00b-49eb-a4ac-b31f02d7f127 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.393225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:16.404170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.405225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.405225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.405503 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.406285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:16.408175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.408610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a078bdc-f983-4209-8308-7a855dd5de3a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.409313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f65b8-1a2d-47bf-8b6e-d14eb7aacaec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.422527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 234/472] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:16.424265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b700-8135-4c0b-a3a1-54987b82850b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.424541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b700-8135-4c0b-a3a1-54987b82850b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.424964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b700-8135-4c0b-a3a1-54987b82850b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.426668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 235/473] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:16.428990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05b700-8135-4c0b-a3a1-54987b82850b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.476969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.478017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.478491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.478951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.479881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:16.480410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.482091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815287-b6d5-4964-90e8-8f8ca6c854d9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.500665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 239/474] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:16.510901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.511514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.512092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.542450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:16.544807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:16.570667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05b700-8135-4c0b-a3a1-54987b82850b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.609680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.610388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.610704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.610990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.611722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.613253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.638996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 240/475] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:16.639731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.682521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:16.778879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c890-edde-4f88-b68c-f23eecc2a6ca None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.785900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 236/476] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:16 2022] POST /identity/v3/users => generated 249 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:09:16.787471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:16.850929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:16.854083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:16.854675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:16.855387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:16.857147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:16.861379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:16.889839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:16.956369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0cea-67fa-43d7-81cc-000ddfeab59c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:16.991873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 241/477] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:16 2022] POST /identity/v3/users => generated 248 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:17.007786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:17.008930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:17.009460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:17.010185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:17.011323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:17.013059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:17.045541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.sql.core [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 14:09:17.045541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 14:09:17.045541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [parameters: {'id': '897a3758fdc84e5f8ee5febe4cfbafa8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 14:09:17.045541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80328) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application result.read() Jul 14 14:09:17.064103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:09:17.065174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application yield resource Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise self.value Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application session.commit() Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 14:09:17.066195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self.session.flush() Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application compat.raise_( Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 14:09:17.067046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:09:17.068474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application result.read() Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:09:17.070769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [parameters: {'id': '897a3758fdc84e5f8ee5febe4cfbafa8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:17.072879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 14:09:17.074895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:17.077708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 237/478] 10.222.0.93 () {60 vars in 1131 bytes} [Thu Jul 14 14:09:16 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:17.077893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:17.184127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:17.202122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71977a-a1cf-4f12-ba4f-c26f867be03f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:17.246059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:17.246883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:17.247167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:17.247445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:17.261501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-132038b0-6c6d-429b-a425-faea642ff2d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:17.269592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:17.270579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 14:09:17.271376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:17.272473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 242/479] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:17.277056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:17.395636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85130a4-cc40-4075-877d-6d9af3afc458 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:17.416516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:17.442472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:17.443060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:17.443342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:17.443619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:17.444686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:17.444996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:17.446407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbe781-b31e-4d01-9400-f1d41ff83753 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:17.466306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 238/480] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:18.090100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7af-5c82-4fa4-969c-9e3e32ed2ed4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.090614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7af-5c82-4fa4-969c-9e3e32ed2ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.090792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7af-5c82-4fa4-969c-9e3e32ed2ed4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.092367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 243/481] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:18.098478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.098969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.099043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.127719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:18.128955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:18.227448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 239/482] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:18.250145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee7af-5c82-4fa4-969c-9e3e32ed2ed4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:18.395509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee7af-5c82-4fa4-969c-9e3e32ed2ed4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:18.433389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:18.434631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.435122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.435597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.447323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-55fe54d1-c6cf-454a-871a-5ce33cdd3e85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:18.455870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:18.456751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:18.457551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:18.460297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 244/483] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:18.469006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:18.559332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6dcb-5c39-4344-8c6e-bf68e4e2f510 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.559986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6dcb-5c39-4344-8c6e-bf68e4e2f510 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.560488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6dcb-5c39-4344-8c6e-bf68e4e2f510 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.562360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 245/484] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:18.575387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.576002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.576488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.589693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:18.601948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f73c9-927d-408a-93b9-2f2dbc0c1487 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:18.601948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:18.611272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:18.612832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:18.638327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:18.639007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.639292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.639572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.640447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:18.640760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:18.642763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:18.656491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 240/485] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:18.658670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:18.770016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 246/486] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:18.805242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:18.912045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391e7d8-d3f9-40df-b010-c81c6eca3ae5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:18.936798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7dbb8-8a12-48b8-86c0-abb0a599c54c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:18.978913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:18.979718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:18.980002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:18.980280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:18.987430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.001968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:19.002629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.003442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.003737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.004023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:19.010128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:19.011027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:19.011811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:19.012433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 247/487] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:19.021335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.026432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.027911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:19.055605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:19.057709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:19.059859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:19.059859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:19.059859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:19.059859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:19.059859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 241/488] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:19.074383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.133361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.157086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e183f5-4c86-45e9-82aa-b6d4ac801917 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.199025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.199625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.199907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.200184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.200942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:19.201251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.205590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.225001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 248/489] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:19.230797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.275509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840a13-5cd9-4486-8aa7-3e391de1eb6a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.350196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.350884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.351170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.351442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.353378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:19.353705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.355199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.358932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.380742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 242/490] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:19.383275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.441690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2db4c-3653-4763-a1c5-fb6d40657cc6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.497907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.498714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.498997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.499278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.515234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.516735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:19.535476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:19.536406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:19.537326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:19.537326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:19.537326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:19.537326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:19.541644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 249/491] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:19.542175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.542998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.575037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e923e-ab4c-450e-a17e-b918f13873c5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.618635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.619323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.619643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.619892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.627986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.629428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-42d64c20-1ffe-4a9f-987d-8865917ddc4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:19.645537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:19.646668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:19.647607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 243/492] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:19.648456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415df2d-0365-4902-8b18-1054e59401cc None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.648748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415df2d-0365-4902-8b18-1054e59401cc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.649144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415df2d-0365-4902-8b18-1054e59401cc None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.654634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 244/493] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:19.654634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a1570-f98f-4b68-9a48-fd4515cfd335 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.654634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a1570-f98f-4b68-9a48-fd4515cfd335 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.654634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a1570-f98f-4b68-9a48-fd4515cfd335 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.654634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 245/494] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:19.655033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a1570-f98f-4b68-9a48-fd4515cfd335 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.702711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f33149-ac80-4ff1-a8bd-d4506ecd399f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.712369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.742787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.743451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.743734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.744014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.745211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:19.745525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.747822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dfd1d-f718-487c-a934-1cc75da2b301 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.766210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 250/495] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:19.766210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.766210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.766210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.778721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a1570-f98f-4b68-9a48-fd4515cfd335 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:19.787781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:19.789005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:19.827910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:19.828703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.829014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.829292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.830179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:19.830491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:19.831949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbac8f-56c4-46af-a442-33f62d7d3219 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:19.850638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.872467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 246/496] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:19.873753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:19.885059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:19.885059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:19.885260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 251/497] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:19.885260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.898896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:19.901303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:19.979886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 247/498] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:19.980758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:19.995420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:19.999568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a09844-8d38-4db6-85c7-f68176a26492 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.039713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.040893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.041506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.042028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.049989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.051794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:20.069256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:20.070254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:20.071147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 252/499] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:20.072346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.119276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcd04e-c933-4655-b94e-2ef277a8750d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.155666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.166307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.170954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.170954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.170954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.206527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51fb7-0274-4f80-af35-370cfba3109e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.256356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.258360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.258523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.259444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.259880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.260303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.276698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:20.283050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:20.283949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:20.284746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:20.289019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 253/500] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:20.290006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.311636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.377328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 248/501] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:20.377328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.403168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92936865-8e2f-4076-afbb-0dc1ab120231 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.403542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.439895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.440696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.440983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.441329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.452332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:20.460617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:20.461606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:20.462458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 254/502] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:20.463112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.488502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59897047-2946-490c-ab6c-10682c3767d5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.530688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.531364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.531648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.531928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.533096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:20.533224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.534691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.555471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 249/503] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:20.556329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.574375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.603721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031e3a-c109-4f3f-b1e7-d5a90d6f4fae None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.639162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.639835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.640118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.640396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.641156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:20.641521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.643468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dec81-dfa3-4a15-992f-efb5a99c0919 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.652106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 255/504] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:20.653479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7adb10-b00e-41ce-851d-4f0596197476 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.653790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7adb10-b00e-41ce-851d-4f0596197476 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.654094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7adb10-b00e-41ce-851d-4f0596197476 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.655594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 256/505] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:20.658663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7adb10-b00e-41ce-851d-4f0596197476 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.666537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.683338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b80f9-03e7-46c1-b3d6-4566abd72c4b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.719153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.720001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.720234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.720516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.731198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.732685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:20.751675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:20.752619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:20.753462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:20.753462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:20.753462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:20.753462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:20.753462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 250/506] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:20.753730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.775066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7adb10-b00e-41ce-851d-4f0596197476 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.811907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.812495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.812786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.813067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.814784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:20.815106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.816512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.825331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 257/507] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:20.826211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:20.828893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.862462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be721643-dd18-4e81-8c43-dfb02f88fae7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.898661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.899279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.899567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.899850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.903406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:20.932534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0743ff-137f-4875-afd8-ad46d180432f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:20.971308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:20.972110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:20.972399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:20.972675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:20.981109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.982625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:20.983696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:20.985634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-409296c3-8dce-487e-b51a-841a04c3b8a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:21.003479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.004413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 258/508] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.010030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.025084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:21.026336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:21.083881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.092747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 251/509] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:21.092747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.101221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 259/510] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:21.102111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.129762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.200884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68fca-79fd-4131-ac6b-5c4ca461522a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.208410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb061b71-e6ca-4e6e-a49a-e423fe61d955 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.239074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.239742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.240024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.240305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.241171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:21.241490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.242913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.244504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.245313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.245601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.245918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.254578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 252/511] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:21.255423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.257177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.258706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:21.277606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.278661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.279518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:21.279518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:21.279518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.279518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:21.279518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 260/512] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.279789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.316693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.359525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.366911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38583c2a-eb3e-49e7-bb8b-1dc89e0eeebf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.383172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1516-49fa-420b-b7c4-7d999f8eca9b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.402017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.402692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.402977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.403253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.403987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:21.404297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.405691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.418031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 253/513] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:21.420255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.421059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.421341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.421445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.421622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.434861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.436458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:21.454827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.455715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.456567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:21.456567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:21.456567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.456567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:21.456567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 261/514] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.456837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.524571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c86974-f292-495f-8a26-e9f2e5f61ec0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.558585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.564837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.565676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.566001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.566266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.571716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357987f-dd37-4487-81e7-133711e10c3d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.579129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.580723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:21.602777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.603843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 254/515] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.605064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.618989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.624349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.627512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.627798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.628075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.639124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.643064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:21.655886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.656801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:21.657649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 262/516] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.658170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.722942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3924669-e628-41ff-836f-2016a6ae3f65 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.760273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.760941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.761231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.761352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc3f6-5c3b-4660-a44c-07dfbb4ab999 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.761516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.762284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:21.762597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.763990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.776011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 255/517] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:21.776843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.799778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.800622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.800909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.801192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.808928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.810408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:21.826133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:21.827106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:21.827984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 263/518] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:21.828557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.840942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.851200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:21.891172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9278-0e06-4da9-9693-cda07510c378 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.929459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.930326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.931113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.931392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.932126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:21.932434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.933909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.940221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a9383-5379-4fb6-bd0d-8bc09d3c0f8a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:21.947619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 256/519] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:21.948684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:21.977798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:21.978467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:21.978753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:21.979026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:21.979753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:21.980062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:21.981431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:21.994178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 264/520] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:21.995021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.054522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44553e09-d328-41ce-b192-081029b2c901 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.079972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.083036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.090857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.091518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.091802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.092079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.092873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:22.093181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.095175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.113681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 257/521] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:22.114553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.114762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e55-d0df-472c-b29a-c10246e1fb96 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.150344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.151062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.151347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.151627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.159865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.161308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:22.176033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:22.177141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 265/522] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:22.178085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.233360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bd3c-b096-4615-9fdd-6f8ffa342d06 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.279446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.280132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.280417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.280697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.281559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:22.281967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.283996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.290972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c669e-cf46-492c-bd6c-682742ae4aba None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.301591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 258/523] 10.222.0.93 () {56 vars in 1139 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:22.302457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.319279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.327574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.328377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.334571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.334571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.337051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.338738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.351238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:22.354233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:22.355084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:22.355932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:22.360316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 266/524] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:22.361126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.414031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b46ef3-f944-4937-928e-f5ed09ab22c1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.459216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.459898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.460186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.460466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.461187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.462723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.482117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb8b8-b577-4f0b-8d36-4365839963ec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.516269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.517443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.517734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.518049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.518900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:22.519221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.520679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.527494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 267/525] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:22.528316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.595564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.628779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 259/526] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:22 2022] POST /identity/v3/users => generated 247 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:22.629996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.661826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5858e-5261-4365-b62b-9c52fe2e0bbd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.731660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.732688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.733234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.733741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.735099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:22.735658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.737383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.784732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 268/527] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:22.786133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:22.806796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4535df-e593-4dbc-b4e5-9ccbfe0d0f39 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.808801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.855872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.857621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.858135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.858565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.925234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d839c97-d1be-4da2-8bc8-0679180104f2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:22.937804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:22.961497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.963016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:22.974584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:22.975613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:22.976132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:22.976632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:22.977645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:22.978207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:22.979918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.000768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 269/528] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:23.002119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.075075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 260/529] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:23.075968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.086188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.117720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ace1b-d88c-4bee-ae72-26e4ae5a6f7d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.155431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.156474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.157027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.157512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.191491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2cfee-efea-41f5-9cec-68c09e14f1d8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.204571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.228633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.229302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.229598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.229910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.230764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1235df33fd6f4917850cf0f2aa850b84) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:23.231081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.232593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.237411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.238925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.240910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 261/530] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/role_assignments?user.id=1235df33fd6f4917850cf0f2aa850b84&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:23.241801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.338546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.360616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 270/531] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:23.363124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37febae5-6bf1-4328-90b9-be9ef9c1da3b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.366536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.403835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.404650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.404937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.405219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.435356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.436875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.448579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:23.462740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:23.464283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:23.465177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:23.465177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:23.465177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:23.465177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:23.465177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 262/532] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:23.470223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.503980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ffac9-915e-4840-b56e-70b907d87991 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.581752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.583154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.585164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.585854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.620377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.620377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:23.636991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:23.638044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:23.638963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:23.638963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:23.638963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:23.638963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:23.640884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 271/533] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:23.647177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.715288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ef2-c1f3-48c0-aac9-5a6dd69670a3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.752431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.783119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.785371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.786053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.786520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.788600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:23.789324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.792537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.816495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 263/534] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:23.823425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:23.856569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a28db7-d886-420f-ac10-882adf2585a2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.911768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:23.935732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:23.936427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:23.936724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:23.937010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:23.937779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:23.938664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:23.940212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:23.956117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0f9c-0a90-4205-9b8d-212eebb7783e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:23.960215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 272/535] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:23.966836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:24.007525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.008460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.008752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.009038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.034117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.035649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:24.056454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:24.057439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:24.058474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:24.069555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 264/536] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:24.070540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:24.136116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:24.156222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93dfac-304b-46ee-a98d-333c9c085d9c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:24.204781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62037cad-2051-491b-a387-8d9303a2d142 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:24.230503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.231721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.232214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.232703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.247163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.247838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.251630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.251630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.251630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:24.251630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.252092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.252168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.252636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.265260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 265/537] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:24.278593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:24.279494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:24.281057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 273/538] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:24.282013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:24.297683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:24.394546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:24.450212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:24.454890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725acb98-e421-48ac-94ce-24304f132ba2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:24.493481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.495744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.496032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.496312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.497165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=937f40c46a3a4c97897706bbc3425afb) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:24.497484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.498993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.515208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 266/539] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:24 2022] GET /identity/v3/role_assignments?user.id=937f40c46a3a4c97897706bbc3425afb&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:24.521102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99896-9c75-4ba1-a431-bd77da74d913 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:24.567681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.568369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.568920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.569237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.570251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:24.570582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.572035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d87393-fec7-4946-9d14-9b45a886699b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.593402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 274/540] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:24.604174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:24.728968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5d8b8-579f-40fc-b6a2-e0b77028d0a9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:24.767183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:24.767854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.768144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.768423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.769268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=398d1954cb704f3898d3984f5c6102a7) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:24.769581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:24.771092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a125-38c9-453a-ac2e-fcd3f6d9e8d5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:24.777646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 267/541] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:24 2022] GET /identity/v3/role_assignments?user.id=398d1954cb704f3898d3984f5c6102a7&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:24.973107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ede3-ae7b-4247-a4ca-3d09e57a9001 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.973485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ede3-ae7b-4247-a4ca-3d09e57a9001 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.973683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ede3-ae7b-4247-a4ca-3d09e57a9001 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:24.975385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 275/542] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:24.981557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bc6fe-63f1-4966-a9bb-88d475836960 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:24.982173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bc6fe-63f1-4966-a9bb-88d475836960 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:24.982672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bc6fe-63f1-4966-a9bb-88d475836960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.009889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1e6bc6fe-63f1-4966-a9bb-88d475836960 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:25.011433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1e6bc6fe-63f1-4966-a9bb-88d475836960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:25.046676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:25.131572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 268/543] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:25.136936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd31bc-4627-4956-90ec-04ad2c37a849 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.137235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd31bc-4627-4956-90ec-04ad2c37a849 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.137513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd31bc-4627-4956-90ec-04ad2c37a849 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.139163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 276/544] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:25.145295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83176c-0369-456a-adf4-613eec46559e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.145593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83176c-0369-456a-adf4-613eec46559e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.145910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83176c-0369-456a-adf4-613eec46559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.157509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd31bc-4627-4956-90ec-04ad2c37a849 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.171196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-cb83176c-0369-456a-adf4-613eec46559e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:25.172991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cb83176c-0369-456a-adf4-613eec46559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:25.303429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 269/545] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:25.339835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd31bc-4627-4956-90ec-04ad2c37a849 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:25.342205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:25.344167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83176c-0369-456a-adf4-613eec46559e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.405300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:25.406140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.406423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.406700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.437408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:25.446457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:25.447397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:25.448186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:25.449614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 277/546] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:25.455942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.526139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:25.552728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83176c-0369-456a-adf4-613eec46559e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:25.642638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:25.643371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.643664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.643941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.651452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dca636-ea61-45c8-b474-0c0f14417125 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:25.674713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:25.682511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:25.684085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:25.685365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:25.689972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 270/547] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:25.695565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.712034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:25.712721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.713003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.713281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.714142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:25.714453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:25.715865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2304aa-7041-4d2a-988a-99dd8c351b10 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:25.725987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 278/548] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:25.727362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8604-1922-4555-9dc3-da557ea8748a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.727653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8604-1922-4555-9dc3-da557ea8748a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.727930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8604-1922-4555-9dc3-da557ea8748a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.729489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 279/549] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:25.734258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c8604-1922-4555-9dc3-da557ea8748a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.762748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:25.810693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7394b-c115-4941-935b-d74b7b128f74 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:25.848312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:25.848998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.849278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.849558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.850360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:25.850468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c8604-1922-4555-9dc3-da557ea8748a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:25.850676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:25.852234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f9492-0222-4353-aaa3-8e93857c11e1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:25.862329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 271/550] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:25.863799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.864097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.864382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.882952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:25.884280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:25.886464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:25.888418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:25.889279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:25.889563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:25.889924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:25.905252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:25.905252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:25.924632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:25.927041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 280/551] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:25.930295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:25.972194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 272/552] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:25.973089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.046342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.081607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0cff9-23f5-4ec2-a87e-e3cd72f1f324 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.106380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a024f-2422-4da1-b423-ce2b2013e1e5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.133557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.134943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.135243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.135531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.146731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.147742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.148174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.148727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.149609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:26.150105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.151232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.151999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.152716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.161121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.167687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 273/553] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:26.168990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:26.181741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.182715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.183573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:26.183573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:26.183573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.183573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:26.185650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 281/554] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.186872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.307521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaabf7-b399-4c29-96d9-644b7c517c9c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.310753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b39-5c15-46e2-a5fc-750d5b17c3c5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.344437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.345895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.346409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.346698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.346979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.355651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.356833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.357318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.358223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.359614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:26.368272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.369225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:26.370694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 274/555] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.371390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.372223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.374047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:26.407956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.410332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:26.412681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:26.417329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 282/556] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.419204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.452849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.516610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06815-b9ba-4bb6-8e69-73bb8d14ddd1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.543194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef8cb-4ded-4369-b023-18f08d573bbf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.557814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.557814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.557814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.557814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.558234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:26.558502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.560014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.574900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 275/557] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:26.576232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.591728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.592619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.593009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.593401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.594919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:26.595220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.596710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.606374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 283/558] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:26.607534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.670159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.699920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e0599-4695-4f90-b454-b1a5dea55815 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.723594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.726045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd75a7-68f7-421d-b6e6-e4eaadcea623 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.744429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.745637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.745909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.746368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.747373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:26.747994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.749681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.765229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.766694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.767052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.767409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.778457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.778457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.778912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 276/559] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:26.780110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:26.794711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.795742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:26.796921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 284/560] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.797619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.908167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1145f-bbc3-4d5a-8082-ed9fdc4f83bc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.913874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925a95-8b7c-401e-83dc-c25bc2aace20 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:26.925748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:26.946983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.948027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.948554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.948976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.952693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:26.954178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:26.954571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:26.955092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:26.962007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.963744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.967470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:26.969687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:26.983548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.984691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.985584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:26.985584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:26.985584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.985584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:26.987773 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 277/561] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.988723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:26.990182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:26.991015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:26.991884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:26.991884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:26.991884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:26.991884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:26.998619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 285/562] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:26.998619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.030538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.112783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29987-c5c8-49f4-83a5-d1bd2a33bd93 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.119567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307b63-d302-475d-bb5b-a956ca406546 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.154415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.155191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.155481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.155758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.156657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:27.157086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.158837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.162739 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.163637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.164107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.164567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.165480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:27.166017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.167718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc2769-9133-4a72-a910-4441eb8b6ba7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.181972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 278/563] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:27.182837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.183196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 286/564] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:27.184783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b0-8c92-41ef-ba83-b15071f1d6f4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.185244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b0-8c92-41ef-ba83-b15071f1d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.185794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b0-8c92-41ef-ba83-b15071f1d6f4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.187235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 287/565] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:27.188685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13b0-8c92-41ef-ba83-b15071f1d6f4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.214200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.278138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.296083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597915-9b36-4849-b5cd-dbf8ead9000d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.305309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13b0-8c92-41ef-ba83-b15071f1d6f4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.336352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.337085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.337327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.337614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.338488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:27.338798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.340360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.343311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.343998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.344286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.344573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.352624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 279/566] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:27.353992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.419161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.420607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35672dba-61dd-45ef-a3d0-43d7c37acc8a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.445866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.468210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69901435-f5a3-4012-ae5f-53b19f31ff08 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.503246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.504037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.504320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.504606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.513200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.514785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.524711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 288/567] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:27.526125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d92c-d2a7-481d-8a7a-a125a49e7d80 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.526499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d92c-d2a7-481d-8a7a-a125a49e7d80 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.526747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d92c-d2a7-481d-8a7a-a125a49e7d80 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.527999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 289/568] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:27.529042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.529451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.529619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:27.531352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:27.532285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 280/569] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:27.537561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.545986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.551583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:27.552777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:27.636936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 290/570] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:27.638378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.656968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2616e1f-ee88-4d29-9b85-4b2705e41e6f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.738935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.739730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.740077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.740281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.747709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.749153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.754564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-26da1378-3e26-4478-80df-db52c79611aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:27.763863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:27.770834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 281/571] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.771827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.772424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.786570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:27.787872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:27.808999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc074-acb8-49cd-ab5f-081439eafa2c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:27.847224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:27.848031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:27.848318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:27.848601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:27.852953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:27.859431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:27.860951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 282/572] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:27.861757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:27.862284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:27.882747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:27.884131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:27.885010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:27.885010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:27.885010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:27.885010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:27.885010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 291/573] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:27.885278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:27.979479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbe891-22a2-496a-9db6-30c62203bd82 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.001137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28934703-cb9b-46e4-b3aa-7c90198c577b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.003452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.014272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.014948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.015779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.015779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.017022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.017337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.018805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.036923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 283/574] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.037801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.043966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.044796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.045084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.045361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.056305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:28.064763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:28.065888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:28.066811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 292/575] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:28.067773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.103498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.161646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791db3-e77c-4d35-81d0-833ba06a069c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.178456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4f30-db67-4008-8f92-a10e29bd4edb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.195510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.196172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.196454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.196731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.197520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.197870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.199314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.216880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 284/576] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.216880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.217134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.217134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.217134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.217134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.226942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:28.234641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:28.235681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:28.236549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 293/577] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:28.237249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.262106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.324633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa0ea0-5c98-4a7a-a8f4-7f5647755673 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.343682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18f39-335e-4b83-a03a-5041b0789298 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.346664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.366376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.367086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.367344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.367622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.368350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.368759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.370249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.377656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.378486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.378777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.379059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.381635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 285/578] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.382591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.390468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.392459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:28.411773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:28.412784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:28.413695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:28.413695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:28.413695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:28.413695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:28.413695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 294/579] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:28.414036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.492446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.496486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f86f0-32c5-4f00-8a94-78d886d61035 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.518512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c052d-b27a-408b-9143-64c9a1da552e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.541305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.542002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.542287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.542558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.543306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.543612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.544975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.553030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 286/580] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.553917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.554043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.554865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.555165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.555434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.578820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=47007139f45e4e55b5a9fa2d2901ef9c, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.580327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.591729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.661046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 295/581] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:28 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:28.662061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.676053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca677c-838b-441e-9419-f35082b64a94 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.723107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.723795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.724075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.724350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.725111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.725760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.727000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.746600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 287/582] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.746600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.800473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05a2e-85b0-4e53-8e8e-8c24bdd36796 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.842265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.854946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.855794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.856110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.856373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.871578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.873516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.883718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:28.887853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ef512-371f-45f9-94bc-18d70e62b171 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:28.890318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:28.891279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:28.892209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 296/583] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:28.892703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:28.941854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:28.943058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:28.943771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:28.944233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:28.945537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:28.946249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:28.948980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:28.971571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 288/584] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:28.971571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.016884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb366f0-8168-4d11-828e-7653dacba21e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.054989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.056000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.059032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.059032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.072251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.073715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.098658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47984-c284-4a23-9faa-da63d428f1c2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:29.101746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:29.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:29.103682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:29.103682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:29.103682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:29.103682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:29.103899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 297/585] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:29.105158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.142947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.143743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.144020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.144297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.156410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.158029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.168540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:29.177340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:29.185963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 289/586] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:29.186890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.194896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.250886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396bfb-dcf8-4a5a-a428-1b9d9dbad948 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.290992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.291662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.291946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.292218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.293045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:29.293356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.294931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.301854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4deb0-0a78-498b-974c-bb7f64ae8934 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.309284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 298/587] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:29.309837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.345898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.346706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.347264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.347264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.355070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.356534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:29.371942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:29.372924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 290/588] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:29.374491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.424406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.429440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bf2a-dc62-4b0b-ac11-2bfe04424c69 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.451064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.466748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.467421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.467713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.467988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.469244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:29.469552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.470995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.485513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 299/589] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:29.486396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.492225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b23ad1-c380-4d48-bb80-c8cf8c2e5d43 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.529192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.529888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.530178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.530459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.531749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:29.532182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.533561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.551828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 291/590] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:29.551828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.608579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b9d8c-42da-462c-9955-04b2365b03e1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.649699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.650740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.650740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.651283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.652130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=4d4c712ae2b04fc68eed213213452664) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:29.652447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.654075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.670119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 300/591] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/role_assignments?user.id=4d4c712ae2b04fc68eed213213452664&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:29.670119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.709387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bd19-1e10-45d4-8c38-2040571c1af3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.714921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.748106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:29.758597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.760367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.761725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.762038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.763505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:29.763872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.766827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.783938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 292/592] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:29.784969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.836569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f3052-e203-43a8-b11a-66a5d504e3f0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.875649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.876788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.877281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.877825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.886567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.888237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:29.904265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:29.905481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:29.906608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:29.907721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 301/593] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:29.908663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:29.922692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864bb8b-3025-4d03-a6e1-65799cfcff5e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:29.979911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:29.981435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:29.982130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:29.982668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:29.994675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:29.997881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.014568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:30.020025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:30.021049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:30.022025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:30.026724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 293/594] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:30.027715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.042166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cc8f7-d819-4c83-b288-c0460c543a3d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.083299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.084597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.085061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.085559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1235df33fd6f4917850cf0f2aa850b84/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.098124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.103152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=1235df33fd6f4917850cf0f2aa850b84, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.104923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.152790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8484104-ba18-4306-85e2-eb26e76e5b01 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.189614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.190477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.192423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.192423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.192423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:30.192423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.193962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.199630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 302/595] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:29 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1235df33fd6f4917850cf0f2aa850b84/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:30.200852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.217064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 294/596] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:30.227199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.333517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b38ca-6017-424b-9a4a-8422d416a7fb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.372180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.381222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.385089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.385863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.386341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.387474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:30.393545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.393545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.411711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 303/597] 10.222.0.93 () {56 vars in 1139 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:30.419109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.448783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25565d-96db-4be0-a77f-4c8f7d13f757 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.467393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.494889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.496340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.496954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.497417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.508687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.510562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:30.530566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:30.531584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:30.532559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:30.532559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:30.532559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:30.532559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:30.534997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 295/598] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:30.540753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.569048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f443-10af-46af-a86c-2dd1592625ac None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.607820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.608937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.616844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.616844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.624402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.626900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:30.678847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:30.679934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:30.680931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:30.680931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:30.680931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:30.680931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:30.683315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 304/599] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:30.688650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.694451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f04a-9c34-45ec-8fec-aef60b179dbe None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.738180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.738883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.739179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.739465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.740199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:30.740512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.742115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.756359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 296/600] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:30.773818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:30.800175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.822169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:30.897650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1e920-da9e-4fe7-a026-40acd19dcd83 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.937989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:30.939150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:30.939663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:30.940149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:30.948509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:30.950334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:30.955930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf71a7-a2db-4138-b456-3a1a655df039 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:30.965636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:30.966617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:30.967500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:30.970799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 297/601] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:30.978968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.055141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.055839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.056109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.056387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.057128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:31.057438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.058882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.072256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:31.089290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 305/602] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:31.095381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551e01-2f5b-4930-981f-2f5b19150757 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.098785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.155396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.156703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.156998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.157279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.158112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:31.158436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.159895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.187675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 298/603] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:31.193840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.216778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5744bf3-89e0-4c34-aeda-288891e3e353 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.218358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:31.255189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.256638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.256944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.257229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.264729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.266366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:31.281278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:31.282266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:31.283147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 306/604] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:31.291504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.369203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31993f-92da-4cee-8811-eda48771d122 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.415453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.417376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.417676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.417988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/398d1954cb704f3898d3984f5c6102a7/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.431840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:31.437847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=398d1954cb704f3898d3984f5c6102a7, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.439425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30115d0-efdf-4526-a0b2-e2f817f478d5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.455009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1b9e-dd95-461b-a266-67aba7cd1516 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.492298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.493012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.493300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.493575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.494452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:31.494758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.496183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2d8de-1fbd-4e14-ab4d-fb32b6b37523 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.515108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 307/605] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:31.516578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4e78-0821-4804-85ce-f671e0f49991 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.516876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4e78-0821-4804-85ce-f671e0f49991 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.517226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4e78-0821-4804-85ce-f671e0f49991 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.522606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 308/606] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:31.522606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f4e78-0821-4804-85ce-f671e0f49991 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.549377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:31.578434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 299/607] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:31 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/398d1954cb704f3898d3984f5c6102a7/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:09:31.581048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.581760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.582301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.625928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:31.631228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:31.706382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f4e78-0821-4804-85ce-f671e0f49991 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.769272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:31.771382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.771820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.772153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/937f40c46a3a4c97897706bbc3425afb/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.793164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=937f40c46a3a4c97897706bbc3425afb, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:31.794705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5840c6-a7e6-49f7-a424-4ce6aafc3e91 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:31.797252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 300/608] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:31.830598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:31.864143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 309/609] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:31 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/937f40c46a3a4c97897706bbc3425afb/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:31.961468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56849a8-9606-42c0-91b6-ff43b874d011 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:31.986810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:31.993348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c883f27-bbe5-439d-9dba-c551f77f9503 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:31.993646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c883f27-bbe5-439d-9dba-c551f77f9503 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:31.993959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c883f27-bbe5-439d-9dba-c551f77f9503 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:31.995647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 310/610] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:09:32.001355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.003840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.003982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.003982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.004119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.004119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.004119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.015219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:32.024382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:32.025388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:32.026287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 301/611] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:32.036070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.051084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:32.052336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:32.162877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 311/612] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:32.184609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60778f5f-be4c-4d56-a168-f61668f8e8ae None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.191165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.195690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.234596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.235201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.235489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.235766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.236712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:32.237005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:32.238655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:32.261800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 302/613] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:32.267805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.318388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.347880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654d8a2-62b3-45bf-b636-58e830c09345 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.376278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5d2b8-acef-46a1-aea6-57693c575db4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.399445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.400265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.400552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.400832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.411874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.412680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.412976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.413271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.425304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:32.426898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:32.427576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:32.447328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:32.448276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:32.449175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:32.449175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:32.449175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:32.449175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:32.449175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 303/614] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:32.451670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:32.452612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:32.454101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:32.457855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 312/615] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:32.459277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.460330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.591000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.644514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.648992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2038-e9bc-4d39-b016-c618283a266d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.653662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034c5-ebd1-4e50-9c36-9f6a3e97c241 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.691562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.692319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.692685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.693289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.694184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:32.694561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:32.696023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1f9c-b56e-4b65-a7ce-dddbfa65dbf9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:32.698090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.698992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.699462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.699959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.700922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:32.701415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:32.703033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:32.713803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 313/616] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:32.713803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca097e1-81bf-492c-b892-f895c7d35889 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.713803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca097e1-81bf-492c-b892-f895c7d35889 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.713803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca097e1-81bf-492c-b892-f895c7d35889 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.716495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 314/617] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:32.716959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 304/618] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:32.717882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.718178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.718496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.722273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.744761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:32.746003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:32.833966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb3fd-c5e5-4b5f-b2bd-c380e091841b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:32.836273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 315/619] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:32.837212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.893880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:32.894611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:32.894893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:32.895171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:32.903225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.908576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:32.909993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:32.929698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:32.931085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:32.932144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:32.932144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:32.932144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:32.932144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:32.935842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 305/620] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:32.936796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:32.949588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:32.971544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49bb9-f59f-44b2-a227-1b24b75d2315 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.028094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.029600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.030227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.030874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.046375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.048575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:33.071939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:33.073140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:33.074355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:33.082921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 316/621] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:33.084377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.088140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197567-f140-4440-b14c-ff830ca29881 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.141737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.142843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.143540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.144027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.160511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:33.175591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:33.177032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:33.178268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:33.181453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 306/622] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:33.182506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.216367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9818-626a-4c6a-b916-4f8e318c6ae4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.236207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.261203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.262065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.262444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.262792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.263587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:33.263960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.265632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.274134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.279363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 317/623] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:33.287967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.319436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a47e4-53e1-42a8-b645-83b2a432357b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.365121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.366182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.366475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.366759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.367618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:33.367934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.369581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.395781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 307/624] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:33.396656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.415260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773c52-e390-42d6-9b3b-1bed1aac5f58 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.464194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.465270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.465741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.466390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.467589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:33.468101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.470723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.482386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 318/625] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:33.483240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.516752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d247f-90db-436c-a681-ee7c86ff1981 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.520378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.546600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.555667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.556714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.557170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.557639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.565440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.567314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:33.584439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:33.585723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:33.586908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:33.593689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 308/626] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:33.594888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.595907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2f5b8-4bd0-4ccf-a21c-1826d578064f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.637342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.638051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.638343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.638627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.701659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9a103-4ee2-4126-817f-43201e482e05 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.711637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.713173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.739666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:33.741242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:33.741931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:33.742577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:33.753141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.759461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:33.761202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:33.786784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:33.788031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:33.789113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:33.789113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:33.789113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:33.789113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:33.795716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 309/627] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:33.795716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.836399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 319/628] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:33.837969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:33.856231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:33.948287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0ef39-6dab-4ede-a129-7c0e4c6e8650 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:33.982120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd992e-8cf0-40c3-b1a5-3db2a2e9800a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.011414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.012751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.013229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.013739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.015516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:34.016085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.018871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447e961-9b05-452b-995c-2fc945ad910f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.024597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.025263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.025549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.025855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.026576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:34.026883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.028232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.029839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.047102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 320/629] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:34.047102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.049628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 310/630] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:09:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:34.050426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3a3d0-e31c-4142-93d3-973039412ee2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.050721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3a3d0-e31c-4142-93d3-973039412ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.051007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3a3d0-e31c-4142-93d3-973039412ee2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.052431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 311/631] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:34.052984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3a3d0-e31c-4142-93d3-973039412ee2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.163519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.170567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd266981-54ab-4e39-9815-593f5f04959e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.170828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3a3d0-e31c-4142-93d3-973039412ee2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.214229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.216095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.216793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.217494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] PATH_INFO: `/v3/users/glance` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.221031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.221906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.222197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.223230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.228601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.230252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.236668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.238174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:34.246160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:34.247150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:34.248096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 312/632] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:34.248709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5a7a68fa-3241-4e9a-85a5-9ec25131f945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:34.267664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:34.268877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:34.269816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:34.269816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:34.269816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:34.269816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:34.269816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 321/633] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:34.270311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.270515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.270795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.291316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:34.292619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:34.337127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.360352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391b73b-6f68-4f0c-a660-d813e08b79db None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.361265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 322/634] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:34.362132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.406365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.407175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.407465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.407982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.418869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.420583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:34.437798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:34.438789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:34.439680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:34.439680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:34.439680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:34.439680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:34.439680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 313/635] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:34.440247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.466788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab8512-9342-4ee1-ad98-b30ef55a1b0f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.502956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.503542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.503835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.504114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.504960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:34.505271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.506757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ab8f0-8c3d-4a5b-99c8-e72970374108 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.524722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 323/636] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:34.526093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d80d-266c-4b5f-8969-a7f71caa09cd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.526506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d80d-266c-4b5f-8969-a7f71caa09cd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.526916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d80d-266c-4b5f-8969-a7f71caa09cd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.529882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 324/637] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:34.530633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13d80d-266c-4b5f-8969-a7f71caa09cd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.547128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d66f-f3cb-41d5-9406-ecaf84684789 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.567165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.582885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.583740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.584037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.584317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.585081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:34.585474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.587072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.599860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 314/638] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:34.600719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.639935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13d80d-266c-4b5f-8969-a7f71caa09cd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.673950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.675423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.676229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.676514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.676915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.687993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:34.696540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:34.698526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 325/639] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:34.699536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.719470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7312aa-7648-4331-b354-4ddc005f790f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.753842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.754427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.754712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.754987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.755716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:34.756034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.757416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9cb77-54cf-4f54-a21b-93bd57146d76 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.770684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 315/640] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:34.772136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.772438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.772716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.789871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:34.791092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:34.800791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.807855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502313-8456-4a89-8bb5-9f0626292935 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:34.843661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:34.844308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:34.844591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:34.844864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:34.861561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 316/641] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:34.870863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:34.916366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:34.917852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:34.933431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:34.977934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98ba6-c6d0-44b6-96e7-52259165f6d8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.012892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.013603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.013862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.013977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.014180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.021064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.029895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 326/642] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:35.029895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.030100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:35.037217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:35.038296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:35.039230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 317/643] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:35.039759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.137105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a501ab-5a0e-471e-844f-390b0e2ec180 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.143170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ade1-c744-4d21-977d-401bf31a3d33 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.160400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.176059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.176736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.177019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.177305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.177413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.178089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.178210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.178397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.178496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.178912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.180021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.185949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.187417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.189509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 327/644] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.190498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:35.203281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:35.204454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 318/645] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:35.205549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.252748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.298535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f92e-34ff-4110-8d31-0cce24b953df None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.318358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c400e-018e-4240-91a5-6dc0aa9b19e6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.333919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.335503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.335970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.336374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.349282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:35.354226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.355022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.355310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.355589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:35.357267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:35.358409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 328/646] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:35.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.371117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.372580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:35.391977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:35.393648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 319/647] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:35.394727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.433522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.466307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003cce9-ea39-4d96-9bf5-7b2153aa7092 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.491395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.506419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3842-db52-40d1-b645-cbba97403d7f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.509123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.509816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.510152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.510480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.511307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.511618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.513066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.525860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 329/648] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.526519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.544259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.545063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.545353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.545636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.557041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.558462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:35.576807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:35.578056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:35.579195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:35.579195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:35.579195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:35.579195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:35.579195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 320/649] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:35.579527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.647055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f68d7-3927-480e-93c0-024233f0feea None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.671089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.688890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.690666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.690666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.691106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.692510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.693303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.695696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d27326-abae-4a3e-9340-b9a59a0e9467 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.696001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.717388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 330/650] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.719343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.744242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.745195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.745395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.745674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.746469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.746776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.748209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.755985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.761131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 321/651] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.761131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.829485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0c49-42dd-4511-9d92-80a2da6ae731 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.868683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.869461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.869760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.870072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.870985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.872746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.872746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.872961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ed10-2fef-4e87-a595-99a65290640c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:35.886422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 331/652] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.887125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.915490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:35.916173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:35.916456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:35.916811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:35.917979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:35.917979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:35.919404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:35.925960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 322/653] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:35.926763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:35.953685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:35.998147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.006191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f363e-ceff-4327-873b-2b7a8424f600 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.059382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.060706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.061722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.061722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.062645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:36.063057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.064552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.080692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 332/654] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:36.080692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.088837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff987-5834-41c4-9a6d-e7d98c7631ba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.135052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.136729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.136729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.136729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/4d4c712ae2b04fc68eed213213452664/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.155330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=4d4c712ae2b04fc68eed213213452664, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.157345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.213472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25d99-4a47-43cd-98fa-a082146e0d28 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.243543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 323/655] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:35 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/4d4c712ae2b04fc68eed213213452664/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:36.244715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.261267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.262408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.262999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.263278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.276941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.278512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:36.300638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:36.301619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:36.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:36.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:36.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:36.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:36.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 333/656] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:36.303352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.317021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.348733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.401119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d8118-95fd-4436-824b-3955a5503f9a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.453298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.454143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.454432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.454713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.454825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddafd4-09b9-42c4-8bb0-51dec779b690 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.462608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.464080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:36.480031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:36.480994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:36.481903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 324/657] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:36.482393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.506616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.507826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.508322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.508841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.518272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.520552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:36.537260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:36.538303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:36.539258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 334/658] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:36.539889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.631986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f863-ede7-40f8-a23a-af118b77c62e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.632191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.643437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.662146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.663201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.663852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.664236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.665180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:36.665587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.666495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b29420-a868-4552-973c-f0d7d12c1f13 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.667660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.681545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 325/659] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:36.682691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.710390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.711774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.712220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.719173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.720382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:36.720973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.722768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.744541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 335/660] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:36.744541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.832943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea9f9c-cd73-4a4f-a731-e9fdc90d8c6c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.872519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01123-8c23-4fda-ae3c-e6b10f8101f0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:36.875944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.877311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.878052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.878587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.879660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:36.880178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.881923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.906627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 326/661] 10.222.0.93 () {56 vars in 1137 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:36.907803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.914618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:36.920307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:36.921165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:36.922001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:36.922291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:36.930444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:36.931865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:36.948033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:36.949221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:36.950250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 336/662] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:36.954244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:36.968983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.059093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582072-1d04-4bb8-a24b-b469f1134a61 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.101501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69f389-b021-438f-9de6-ef153b95cb14 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.105121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.105927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.106369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.106769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.107731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1235df33fd6f4917850cf0f2aa850b84) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:37.108192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.109886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.125807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 327/663] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/role_assignments?user.id=1235df33fd6f4917850cf0f2aa850b84&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:37.125807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:37.180520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.181196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.181485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.181786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.319217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.328610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.332380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c79f-afcc-44d9-89cc-b8310fbc29f3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.336144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.336591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.392598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.393697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.394241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.394757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.395955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:37.396568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.401564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.429853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 328/664] 10.222.0.93 () {56 vars in 1139 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:37.431972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:37.450995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 337/665] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:37.456837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:37.562921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3c58-21a1-4ff2-a6ac-0998dd959590 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.574321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.587923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3d4c3-a150-4748-bba7-1f9a1698695b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.612291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.613392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.613917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.614404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.636393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.637110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.637403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.637682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.651811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.652859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.654657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:37.677001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:37.678121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:37.679083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:37.679083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:37.679083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:37.679083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:37.680830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 338/666] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:37.688395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:37.724339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.726144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.838220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e70415-64a3-4b38-a68a-afced9f9cf13 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:37.885812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:37.888146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:37.888839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:37.889453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:37.898879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:37.901685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:37.903300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:37.922889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:37.932364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 339/667] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:37.940756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:37.973137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 329/668] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:37.993585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.088044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbf18-5005-4bdc-b15a-3e657bc0d2b3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.143722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.146196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.146769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.147295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.162182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:38.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.167638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:38.198604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:38.200276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:38.201650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:38.205403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 340/669] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:38.209833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260684a-3d2d-4f7b-a3c8-729670a0a3a2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.212145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.247318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.247318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.247630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.247724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.265581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.265581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:38.298456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:38.299460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:38.300349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:38.300349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:38.300349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:38.300349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:38.307270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 330/670] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:38.314042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.352381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:38.447585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19c20c-d5b5-47a1-b40d-915a7efb8178 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.510013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.511201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.511699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.512188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.513209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:38.513883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.516221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1566f-6021-4d4c-8eb9-6ca3aee3be89 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.525916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41415-d146-477d-8ea7-ba4d68987e3d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.536109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 341/671] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:38.538039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367b9d-ced2-49d6-8b52-74d9bbfd88b6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.538729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367b9d-ced2-49d6-8b52-74d9bbfd88b6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.539087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367b9d-ced2-49d6-8b52-74d9bbfd88b6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.540988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:38.541236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 342/672] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:38.546613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367b9d-ced2-49d6-8b52-74d9bbfd88b6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.570712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.571442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.571729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.572006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.572739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:38.573048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.574471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444f66-270c-491b-ac5c-d9b1c2de1a6e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.591188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 331/673] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:38.592508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.592821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.593103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.610902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:38.612189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:38.701813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 332/674] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:38.701813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.705633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367b9d-ced2-49d6-8b52-74d9bbfd88b6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.739047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:38.754133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.754807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.755094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.755372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.756310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=398d1954cb704f3898d3984f5c6102a7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:38.756520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.757967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca465-92ca-49d7-8933-e4ec8839a332 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 343/675] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/role_assignments?user.id=398d1954cb704f3898d3984f5c6102a7&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f95e0-6c77-4a0f-8527-8311d24f75c5 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f95e0-6c77-4a0f-8527-8311d24f75c5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f95e0-6c77-4a0f-8527-8311d24f75c5 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 344/676] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:38.774184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f95e0-6c77-4a0f-8527-8311d24f75c5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:38.844140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:38.848892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c778e-0081-4150-95a6-8026a967a3f0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.905637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.906762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.907181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.907524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.907634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f95e0-6c77-4a0f-8527-8311d24f75c5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:38.917866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:38.920483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-040ba56a-e23d-4fdf-a7c0-9e4a3b457a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:38.942002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:38.943346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:38.944242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:38.945976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 333/677] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:38.949336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.949336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.949336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.956235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:38.957043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:38.957333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:38.957611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:38.978433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:38.980575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:38.981801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:38.987881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:38.990001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 345/678] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:38.992558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.059521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.073987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 334/679] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:39.075175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.141017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2681-cce4-4017-b048-52d9b98810dc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.191022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.192171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.192733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.193016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.194146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:39.194464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.195061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.196190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.205995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb199f-314b-4326-95af-ae60788af8da None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.217344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 346/680] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:39.219117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.250248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.251235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.251667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.252119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.253086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:39.253632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.255245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.265979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 335/681] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:39.267791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.355391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed89896-cfc4-4024-a07e-03dc690c7661 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.359124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.399078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc2020-a91f-452d-b0bd-ba76c5016c7f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.402566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.403622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.404105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.404603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.419288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:39.428105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:39.429166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:39.430115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:39.437914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.438647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.438949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 347/682] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:39.439475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.439758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.440533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.440651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=937f40c46a3a4c97897706bbc3425afb) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:39.440905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.442900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.457883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 336/683] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/role_assignments?user.id=937f40c46a3a4c97897706bbc3425afb&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:39.461267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.501252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.640619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339314d-fff5-4834-9f0d-b3dbc587b491 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.684925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.685743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.686122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.686357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.687139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:39.687448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.688635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f33-9988-47d8-953d-549b14bfb5f1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.688833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.708335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 337/684] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:39.713258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.746562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.747368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.747861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.748654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.766275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.767737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:39.790359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:39.792012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:39.792990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:39.792990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:39.792990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:39.792990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:39.792990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 348/685] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:39.797251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:39.826532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.834454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:39.862414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16dbe-ba5e-4984-88ba-53a4ef397a54 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.915934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:39.916663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.916950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.917224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.933380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:39.935361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:39.950009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc83b-26cb-4f8b-8d46-b634b7e3cfdf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bb7dcf14-24bf-41f4-9290-6cb00820a11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:39.955688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:39.956697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:39.957612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:39.957612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:39.957612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:39.957612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:39.961548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 338/686] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:39.963008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059810-60a2-4c4b-9f36-52272e043c31 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:39.963296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059810-60a2-4c4b-9f36-52272e043c31 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:39.963726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059810-60a2-4c4b-9f36-52272e043c31 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:39.965324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 339/687] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:39.971261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa059810-60a2-4c4b-9f36-52272e043c31 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.005338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.006505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.007006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.007554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.008556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:40.008996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.010815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a738433-6f50-4ef7-a5c1-1b7cba9b617e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.031844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 349/688] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:40.033184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.033534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.033857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.059290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:40.060568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:40.101555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa059810-60a2-4c4b-9f36-52272e043c31 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.136661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.144887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.145677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.146002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.146286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.155146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:40.155146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.155357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 350/689] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:40.155357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.155494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.157339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.180772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 340/690] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:40.186242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.275052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cef37-3080-4486-a8e7-1f576793933e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.318300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.319712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.320176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.320709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.329673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.331569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.335906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924b9e-09dc-4722-9487-c581116d72b5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:40.349781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:40.351854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 351/691] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:40.354243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.382154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.383224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.383522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.383808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.395946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.397416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:40.416747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:40.418823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 341/692] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:40.421387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.432129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.479939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5ca7-029e-4aa7-a29a-9ddbee1f18d7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.497294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.526456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.526837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.527082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.527359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.544419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.547015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.565432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b9ab4-78ea-42e8-a6ab-48869ad3c3e8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:40.574717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:40.575666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:40.576611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:40.576611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:40.576611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:40.576611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:40.580084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 352/693] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:40.581246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.612173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.612863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.613248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.613624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.614607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:40.615017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.616564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.637556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 342/694] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:40.638388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.726986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2646d7a-c37d-4529-9691-4ac978037eb8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.758188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.759974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76a6a0-0b27-4403-880d-a2402a3acc7e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.771003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:40.782752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.784160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.784606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.785321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.786550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:40.787027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.789298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.805490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.806460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.806924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.807355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:40.809019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:40.809490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:40.811386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:40.812688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 353/695] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:40.813703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.829399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 343/696] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:40.830594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:40.941008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5662-a5e4-433e-a5d8-c3707435e458 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.969308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1cf3-3a5e-4364-aeeb-6329a6758256 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:40.993844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:40.995149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:40.995590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:40.996017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.014653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.015376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.015818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.016176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.062274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.063859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.092972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.095146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.103141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.103990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617ed6c-2345-44d9-8df4-4ba11cfcf384 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.215299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 354/697] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:41.215299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.224256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 344/698] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:41.226050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf23e3-eeb6-4df3-abb8-33931ede7c7d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.226364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf23e3-eeb6-4df3-abb8-33931ede7c7d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.226863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf23e3-eeb6-4df3-abb8-33931ede7c7d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.237836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 345/699] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:41.237836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf23e3-eeb6-4df3-abb8-33931ede7c7d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.309195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.316008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.341837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cb3ef-e7ef-4b8d-b9a8-c95af5f0008e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:41.346918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf23e3-eeb6-4df3-abb8-33931ede7c7d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:41.380354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.381080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.381460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.381903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.382650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.384101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081caac-ab87-4178-8073-8b1dbabca5f9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.385110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.385933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.386218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.386495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.401482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.403088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:41.421872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:41.423050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 346/700] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:41.424066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.497791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 355/701] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:09:41 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:41.499340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.499672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.499959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.544013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51268d6-4359-4c92-bb64-2a056771dcb1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:41.547626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:41.548866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:41.577257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.597534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.598642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.599082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.599561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.615626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.618319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:41.641610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:41.642845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:41.644020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:41.644020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:41.644020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:41.644020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:41.649814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 356/702] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:41.649814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.651167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 347/703] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:41.657644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.725398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.795507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d36059-5aaa-4828-8f57-ccd3a093d258 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:41.810543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a67b-3026-46eb-9e82-7852482cbd80 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:41.842961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.844031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.844502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.845023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.846125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:41.846720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.848359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.855177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:41.855844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:41.856131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:41.856408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:41.857712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:41.858093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:41.859965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:41.863327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 357/704] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:41.864398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.885111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 348/705] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:41.886419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:41.937565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:41.992640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74879d2b-640f-4727-9a9c-f51acc53c09f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.019667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abedf3f-f603-4dd1-8ce5-02667168e718 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.021796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.034923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.035751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.036059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.036364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.037038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.038469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:42.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:42.055849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:42.056930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:42.062790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 358/706] 10.222.0.93 () {56 vars in 1129 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:42.064394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.076063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.076877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.077163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.077433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.088654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.090167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b0754725-afa8-41b9-960a-466bec57372d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:42.105885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:42.107314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:42.108774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 349/707] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:42.109309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85074c80-74a3-422c-a61a-095123c2071d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.109309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85074c80-74a3-422c-a61a-095123c2071d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.110627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85074c80-74a3-422c-a61a-095123c2071d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.111114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 350/708] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:42.111890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85074c80-74a3-422c-a61a-095123c2071d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.209307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9115f-ebcc-4afe-a397-97196322f36d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.242324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.248169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.249805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.250218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.250945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.254046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85074c80-74a3-422c-a61a-095123c2071d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.261549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.263138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:42.278732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:42.279998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:42.281184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:42.286713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 359/709] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:42.286713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.305485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.306292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.306655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.307009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.307799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:42.308186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.309691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bdc26-5956-41ef-b6f6-d019744153d3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.323110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 351/710] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:42.323110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.323110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.323110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.338939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:42.340336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:42.349456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.416445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdfb2c-a6e7-4719-883e-90a09fec209a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.433799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 352/711] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:42.433799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.477508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.478325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.478611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.478891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.479701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:42.480016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.481465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.494976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 360/712] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:42.498751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.541753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.564413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d570d-59b9-4fe5-95db-6a7bdbc00acd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.605866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.606893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.607363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.607846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.609165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:42.609667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.612925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.622421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1c098-4cd4-4633-ba51-87ceb80968f1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.627353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 353/713] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:42.628242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.636096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.668390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.669153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.669563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.670027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.671536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:42.673208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.673455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.691895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 361/714] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:42.691895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.761274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118408b1-8c30-4ed6-b88d-b017f8b1d768 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.832220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.833147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.833506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.833888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] PATH_INFO: `/v3/services/network` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.835038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.836435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:42.852470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:42.853787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:42.854832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:42.862018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 354/715] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:42.862018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:42.869152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4d7a-f32c-429b-89fa-db9a3ed77f34 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:42.874584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:42.925061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:42.925647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:42.926008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:42.926289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:42.927317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:42.927671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:42.929266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:42.941797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 362/716] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:42.941797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:43.025299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:43.099707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342aea9-ba18-412d-94e7-d3009af3395f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:43.125655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fad5-283c-4cf5-8d17-ceb53af93c77 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:43.174234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:43.175350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:43.175835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:43.176328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:43.177249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] RBAC: Adding query filter params (name=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:43.178033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:43.178493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:43.179195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:43.179456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:43.179701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:43.181223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=4d4c712ae2b04fc68eed213213452664) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:43.181223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:43.182414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:43.183688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41aec9-80de-4e85-91ab-3a628a033d8f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:43.197813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 355/717] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/role_assignments?user.id=4d4c712ae2b04fc68eed213213452664&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:43.198843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 363/718] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:43.207105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:43.315051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:43.344749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e85aa-6988-4bfd-b12f-f67745d0aab4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:43.398931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:43.399623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:43.399912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:43.400193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:43.400925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] RBAC: Adding query filter params (type=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:43.401236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:43.402663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f19c7-6432-477b-a9a7-22182c61b15e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:43.409413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 356/719] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:43.851300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7194cb-410f-4f06-af4d-a31148b0d369 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:43.851820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7194cb-410f-4f06-af4d-a31148b0d369 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:43.852215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7194cb-410f-4f06-af4d-a31148b0d369 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:43.853942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 364/720] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:43.860259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:43.860613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:43.860902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:43.883269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:43.884503 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:43.985825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 357/721] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 14:09:44.027879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7194cb-410f-4f06-af4d-a31148b0d369 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:44.179021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:44.187480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7194cb-410f-4f06-af4d-a31148b0d369 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:44.243526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:44.244926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:44.245481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:44.245987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:44.266789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:44.275188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:44.276247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:44.277095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:44.279611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 365/722] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:44.285697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:44.416958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:44.464089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61779-6105-4fe3-9fce-52ef264ae80f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:44.513798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:44.515039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:44.515326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:44.515607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:44.516367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:44.516677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:44.518142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:44.529690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 358/723] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:44.535837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:44.724802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11f986-a32e-4ad9-9cd3-8033df3cbf95 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:44.790088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:44.791070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:44.791356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:44.791742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:44.812868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:44.814424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:44.836550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:44.837499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:44.838434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:44.838434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:44.838434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:44.838434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:44.838434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 366/724] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:44.843659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:45.002912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda65df7-9da8-46e3-a99f-a4885f104077 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:45.051635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:45.057923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:45.057923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.058197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.059716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.059716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:45.059716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:45.060855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44340b02-2182-4972-87c0-ed11ec1995b0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:45.081302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 359/725] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:45.086898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:45.119434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:45.257479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0634c2-9c85-425f-870f-44f999b41908 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:45.277731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bd676-4597-483d-828a-746e13521065 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.278067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bd676-4597-483d-828a-746e13521065 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.278330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bd676-4597-483d-828a-746e13521065 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.279971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 360/726] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:09:45.285363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.285652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.285981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.321434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:45.322669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:45.324626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:45.325454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.325735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.326035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.341880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:45.343860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:45.375486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:45.376713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:45.378045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 367/727] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:09:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:45.383778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:45.455005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 361/728] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:45.477626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:45.522503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:45.571273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc50f1e-4781-4c09-b49c-2cd6e7747c36 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:45.618682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794f9c-3b5e-4fba-bf80-70daa62548f4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:45.630791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:45.632173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.632837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.633200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.634566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:45.634891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:45.636822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7872da4-73d9-48f2-bff7-e73af3f0799b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:45.647209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:45.662735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:45.663779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.664279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.664766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.665719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:45.667353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda84648-1ae9-4cf4-a148-68879a67f124 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:45.685633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 368/729] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:09:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:45.687355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc959a0-c469-4f40-9604-2092282408d5 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.687701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc959a0-c469-4f40-9604-2092282408d5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.687974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc959a0-c469-4f40-9604-2092282408d5 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.694242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 369/730] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:45.694242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc959a0-c469-4f40-9604-2092282408d5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:45.767293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 362/731] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:09:45 2022] POST /identity/v3/services => generated 251 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:09:45.767293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f0ff-64b7-40a7-94fb-8531a3054e78 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.767293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f0ff-64b7-40a7-94fb-8531a3054e78 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.767293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f0ff-64b7-40a7-94fb-8531a3054e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.801869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6701f0ff-64b7-40a7-94fb-8531a3054e78 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:45.804976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6701f0ff-64b7-40a7-94fb-8531a3054e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:45.927883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc959a0-c469-4f40-9604-2092282408d5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:45.971906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 363/732] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:45.974876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:45.980338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7f04-cd3b-4aee-999f-efba7c48c6df None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.980630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7f04-cd3b-4aee-999f-efba7c48c6df None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.980910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7f04-cd3b-4aee-999f-efba7c48c6df None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:45.982757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 364/733] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:09:45.993820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:45.993820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:45.993820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.002270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.003276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.003740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.004211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.013689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:46.015494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:46.103216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:46.104062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 365/734] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 14:09:46.105000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:46.105257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.156931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:46.245696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1bb12-89af-46b2-a3c8-d83583d0ff42 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:46.265859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 370/735] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 14:09:46.266993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.296836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:46.300972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.301809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.302104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.302381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.317914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:46.325590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:46.327859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 366/736] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:46.329723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.399191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb049f-6df0-4147-8017-dab10f47432d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:46.444096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.444474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.444759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.445042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.445802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:46.447216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:46.453023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:46.456183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb03bc-959e-4a8b-91db-6114a6cc1f19 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:46.494355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.495186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.495525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.495756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.506736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:46.509990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-38bd664e-9b30-4dbf-b9ea-62aa592880f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:46.529849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:46.530858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:46.531785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:46.531785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:46.531785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:46.531785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:46.533910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 367/737] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:46.535757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e0e67-39f5-4276-9719-27cb9d5b3b70 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.536048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e0e67-39f5-4276-9719-27cb9d5b3b70 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.536327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e0e67-39f5-4276-9719-27cb9d5b3b70 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.538131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 368/738] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:46.538900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e0e67-39f5-4276-9719-27cb9d5b3b70 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.539226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 371/739] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:09:46 2022] POST /identity/v3/services => generated 240 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:46.541838 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.683576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:46.711615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6ffe-2ce7-4c98-b6ab-be10664ef4de None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:46.754821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.755915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.756438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.756924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.757919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:46.758448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:46.760171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4de7-b4f4-49fe-8782-bbab4864daaa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:46.779841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 372/740] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:46.781207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.781485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.781798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.801549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:46.803230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:46.875122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e0e67-39f5-4276-9719-27cb9d5b3b70 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:46.884976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 373/741] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:46.886258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.926704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:46.927822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:46.928170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:46.928557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:46.930140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:46.933006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:46.934796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:46.945434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 369/742] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:46.951135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:46.969551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:46.979179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.024913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f57e6-01a5-4b61-8502-e86de88091f0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.068969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c673-2a02-40f9-8ce6-df1e579dc50d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.075070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.076198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.077255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.077761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.089909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.091753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:47.106597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:47.108036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:47.108928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:47.111484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 374/743] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:47.112557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.128993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.130780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.131255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.131545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.153192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:47.163706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:47.164715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:47.165575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 370/744] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:47.166768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.252873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755dc21-21ac-465a-a602-a9f73ac09d3b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.267212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.284597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.291274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.292316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.292748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.293170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.297684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62c3b-199f-49c4-a940-7872933eb118 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.309188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.312073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.339102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.339824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.340118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.340450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.341591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:47.341717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:47.343395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:47.344591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:47.344591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:47.344591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:47.344591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:47.344881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.344881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.346831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 375/745] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:47.348068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.358634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 371/746] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:47.359419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.472075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8fb39-b4d9-45a0-88fa-6b65ea5d4956 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.472556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fbb-018a-408f-a628-5d915787fe7b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.510769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.511359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.511649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.511919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.512663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:47.512979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.514938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.516669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.517335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.519545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.520215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.520912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.522454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:47.523296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.526100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.529910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 372/747] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:47.531310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.541259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 376/748] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:47.542593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.594412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.653669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12a4c-5bd4-4137-bdbe-89aa19f4e3dc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.662236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa348d-b6eb-4a18-9ca7-d51535218796 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.695902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.696664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.696957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.697235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.698240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1391f1de60b9465295b94c16f8a54872) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:47.698560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.700003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.703785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.704867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.705372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.705872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.706484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 373/749] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/role_assignments?user.id=1391f1de60b9465295b94c16f8a54872&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:47.707393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.717422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.719461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:47.753372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:47.754407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:47.755363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:47.760730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 377/750] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:47.761840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:47.830176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:47.873912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1820f-c8d3-444f-b9dc-e8eae553cfc3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:47.930489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:47.931464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:47.931756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:47.932078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:47.949276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:47.950799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:47.970527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:47.971489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:47.972364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:47.972364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:47.972364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:47.972364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:47.972364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 374/751] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:47.978756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.046485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a8513-4da5-4d2e-a38b-6bbc07d423c1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.049038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.108574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e3a8-2a46-4a9c-9cbf-29287ab75828 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.134203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.141360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.142077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.142365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.142658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.143500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:48.143812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.145268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.177889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.178865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.179154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.179489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.180233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:48.180544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.181957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.188344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 378/752] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:48.194088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.213021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 375/753] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:48.218129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.361823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701586-437b-4df7-8528-9b5b5e36638c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.380502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8a4ad-5c82-47b8-aafb-35a931da1437 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.411895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.413718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.414114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.414404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.430127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.431607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.434520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.435414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.436237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.436564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.445884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.447337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b4bab2ce-f6aa-4138-b96f-f5b29c75edd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:48.453337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:48.454301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:48.455221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:48.455221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:48.455221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:48.455221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:48.459631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 379/754] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:48.461081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe6ea-e520-46cc-8386-8018775fad9b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.461373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe6ea-e520-46cc-8386-8018775fad9b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.461660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe6ea-e520-46cc-8386-8018775fad9b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.463367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 380/755] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:48.464089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe6ea-e520-46cc-8386-8018775fad9b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-655a8238-af9b-4ff1-a62c-d52f55398a12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:48.468216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:48.470646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 376/756] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.472557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.473427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.485802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.511232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:48.513816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:48.547542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.596119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe6ea-e520-46cc-8386-8018775fad9b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.610310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 377/757] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:09:48.611206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.645882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.647046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.647515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.647968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.648867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:48.649329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.651168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.665864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 381/758] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:48.666976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.742885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7594-e74e-4881-9cb0-59d2bf025a75 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.756624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.787782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.788339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70383c63-ade5-4ef2-9dc4-19af10940002 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:48.788506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.788796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.789072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.789959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:48.790268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.792475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.815989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 378/759] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:48.815989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.842533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:48.844521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:48.844521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:48.844521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:48.844889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:48.847788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:48.849891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:48.873947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:48.875857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:48.877528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:48.880432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 382/760] 10.222.0.93 () {56 vars in 1129 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:48.881494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:48.985496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea2247-6b6e-4ccd-865b-de089dc790af None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.028957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507748d0-520a-4377-91a7-ab25a2df8fd2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.029201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.030612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.030898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.031175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.039420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.040876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:49.056091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:49.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:49.058065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 379/761] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:49.058555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.086262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.086960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.087256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.087840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.101469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.158448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.168496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.172518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.190913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f42f-eaea-49c6-b870-59dec78b5b28 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.232028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.232703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.232996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.233274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.234015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:49.234322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.235765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867919d7-1840-4412-8d46-c70e6fa0f78c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.240861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 380/762] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:49.244044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234cad-54d8-4e80-aafd-65633610393f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.244464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234cad-54d8-4e80-aafd-65633610393f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.244869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234cad-54d8-4e80-aafd-65633610393f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.254409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 381/763] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:49.254409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a234cad-54d8-4e80-aafd-65633610393f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.300448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 383/764] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:09:49.300448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.339979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.381800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a234cad-54d8-4e80-aafd-65633610393f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.415468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda018-a78b-46f0-af6f-8d931081bc23 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.422762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.423424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.423707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.423984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.424775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:49.425085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.426553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97762fb7-5ef6-4450-a048-a487403c62b0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.443108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 382/765] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:49.444398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.444688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.444962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.452205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.464876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:49.474146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.474315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.474400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.479435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 384/766] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:49.479435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.549875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 383/767] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:49.549875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.604443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cbcb6-6117-4ce1-9c34-f75350b2d635 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.606479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.644679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.645491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.645875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.646160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.657807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.659559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.679177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16fe0e-3e49-4f47-9740-dd75bd5f041a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:49.679933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:49.680801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:49.682168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:49.682168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:49.682168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:49.682168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:49.682168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 385/768] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:49.682641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.717640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.722626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.724040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.724325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.724604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.726808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=47007139f45e4e55b5a9fa2d2901ef9c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.728266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.798238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9959f-993e-4878-ad59-afeeb6fe9fe2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:49.828695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 384/769] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:49 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:49.828695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.841186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:49.841704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:49.842016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:49.842295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:49.843187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] RBAC: Adding query filter params (interface=public, service_id=877988f461774cc39addb56971519997, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:49.843508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:49.845000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:49.858372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 386/770] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/endpoints?service_id=877988f461774cc39addb56971519997&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:49.859965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:49.898657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:49.982092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecac961-51c1-4fd7-85d0-8c3fac29acf4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.000682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a0df-4dd9-4927-af2e-421257316525 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.042942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.044084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.044612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.044951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] PATH_INFO: `/v3/services/network` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.046228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.047580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.050415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.051081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.051367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.051646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.052375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.052809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.054224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af739f7-75ad-413b-8573-ed7419d3afcd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:50.060332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:50.061915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:50.063996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 385/771] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:50.067262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.073356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 387/772] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.074850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095562f2-366f-406f-8810-a533eda58071 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.075143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095562f2-366f-406f-8810-a533eda58071 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.075419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095562f2-366f-406f-8810-a533eda58071 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.077124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 388/773] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:50.077882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095562f2-366f-406f-8810-a533eda58071 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.134236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.174531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.188168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17da4-f008-4e9d-94e1-68bf8ed09102 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.199397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095562f2-366f-406f-8810-a533eda58071 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.225035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.225705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.226038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.226325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.227015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.228397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.235292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 386/774] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.235937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.245673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.246884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.247172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.247451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.248186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.248496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.250532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961602-9c7e-43e5-8447-e8a12578ebf1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.256458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 389/775] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.257979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.258307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.258557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.277283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:50.278707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:50.362346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 390/776] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:50.363333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.370154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d50ffa-e17e-4cae-bea9-1a9a35e05143 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.409501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.433505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.434391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.434847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.435127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.444972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.446500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.458651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:50.468889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:50.472812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 387/777] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:50.473706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.509895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b46c-e454-4231-bf42-e8f7b2acbcba None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.563619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.564299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.564682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.564969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.565689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.566026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.567777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.573942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 391/778] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.574916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.614086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bed1-fe2b-4089-94b1-4cd373107082 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.663170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.664204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.664498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.664782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.680419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:50.691352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:50.692903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:50.693802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 388/779] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:50.694844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.707451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d715b4-f402-4b2d-9b79-c6aa2bf03ca2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.749218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.749728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.750033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.750310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.751176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.751408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.752895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.756263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.765194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:50.770693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 392/780] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.771556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.845884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7dcb0-08bf-4cea-9623-1956af7ec875 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.893722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.894659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.894945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.895220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.896695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] RBAC: Adding query filter params (interface=public, service_id=ab47788a4eba40b6b7558673dc043db6, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.896695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.898226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.907151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 389/781] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/endpoints?service_id=ab47788a4eba40b6b7558673dc043db6&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.908080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:50.916171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb659c-71d9-44ec-9ad4-64d154bf1b23 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:50.968264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:50.968932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:50.969219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:50.969495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:50.970489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:50.970801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:50.972176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:50.985819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 393/782] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:50.985819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:51.046526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb33f-cb25-472a-81e2-2fbf1d585d4f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:51.050332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:51.074255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:51.103297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:51.104072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:51.104439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:51.104780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:51.105685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=4d4c712ae2b04fc68eed213213452664) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:51.106090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:51.108153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:51.121929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 390/783] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/role_assignments?user.id=4d4c712ae2b04fc68eed213213452664&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:51.123658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:51.137192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec0eb3-19f0-4841-9adb-aeb1b03a21b7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:51.182650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:51.183304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:51.186060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:51.186060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:51.186060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:51.186554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:51.204988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 394/784] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:09:50 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:51.284054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3755-eb5b-4099-90bc-84c9b1e4f061 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:51.324124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:51.324955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:51.325241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:51.325521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:51.338344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:51.339830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:51.361038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:51.363204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:51.365620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:51.365620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:51.365620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:51.365620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:51.365620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 391/785] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:51.369315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:51.506222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d5d47-3286-4d12-a22d-e37d1cc3fcfa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:51.536781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:51.549552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:51.553623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:51.566666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 395/786] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:51.572294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:51.604046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:51.713609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbf03e-0e4e-4982-b377-6fd0adab3305 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:51.763529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:51.764332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:51.764615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:51.764890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:51.774953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:51.776445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:51.799354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:51.804001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:51.805840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 392/787] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:09:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:51.810960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:51.964495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74fb2b-8980-4821-9737-cb23e1e40f55 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.019574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.020643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.021140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.021643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.022736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:52.023259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:52.024930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3f5c-f8ab-4d6d-89ab-66264bea4d6f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:52.052846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 396/788] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:09:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:52.066147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.118130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e50bd-6524-4909-836d-752d957bea0a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.118829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e50bd-6524-4909-836d-752d957bea0a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.119328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e50bd-6524-4909-836d-752d957bea0a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.121098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 397/789] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:52.126206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.126614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.126998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.166219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:52.168967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:52.236849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:52.253795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:52.279835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 398/790] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:52.287821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3517ae-f0c6-42fe-a8b6-32f19881b5b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.334384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.336123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.337214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.337801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.338266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.364634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:52.366611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:52.395209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:52.396165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:52.397250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:52.397250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:52.397250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:52.397250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:52.399659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 393/791] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:52.407666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.468664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6d545-dc9a-44dd-a5bc-26532082d3fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.520740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.521547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.529800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.529800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.534000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:52.543671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:52.545538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:52.546420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 399/792] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:52.550122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.584909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:52.620359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1af9-a753-4168-8e5f-60ed34717bfd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.669061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5b6f7-d675-413c-b59d-e8ead5386806 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.703028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:52.719445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.720128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.720418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.720702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.721449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:52.721823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:52.723178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df756977-f939-4b29-a4dc-5a3330bd7b82 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:52.725692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.727885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.728284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.728673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.729752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:52.730265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:52.732349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:52.745295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 400/793] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:52.753961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.758088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 394/794] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:52.760725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c242a0-2a95-4f25-8d71-3389901362d5 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.761243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c242a0-2a95-4f25-8d71-3389901362d5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.761721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c242a0-2a95-4f25-8d71-3389901362d5 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.768138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 395/795] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:09:52.770357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.771126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.771865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.790400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:52.791961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:52.867395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:52.874145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 396/796] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:09:52.875341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:52.878386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb3a9c-ad70-4aa1-a2b6-c52b11405268 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:52.938449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:52.939609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:52.940123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:52.940609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:52.953532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:52.955404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:52.973507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:52.974597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:52.975483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:52.975483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:52.975483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:52.975483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:52.982037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 401/797] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:52.983279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.001812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd30c-0150-4e3b-a516-51d74e3d9f10 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.026940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.040219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.042249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.042249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.042249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.050005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.051842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:53.067756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:53.068724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:53.069608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:53.070207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 397/798] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:53.071061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.110922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ca77-0d61-4a8c-8299-71f8fb006184 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.146497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.147321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.147603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.147883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.148544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.149936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:53.163428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:53.165649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 14:09:53.167653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:53.177791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 402/799] 10.222.0.93 () {56 vars in 1129 bytes} [Thu Jul 14 14:09:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:53.177791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.183024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.195848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807ce0-0025-4b91-8c63-41ec8de3a0ce None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.234693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.235416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.235709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.235987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.236721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:53.237033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.238437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.251305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 398/800] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:53.252207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.283477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c2a9-5941-4b07-9897-294eb2b65fca None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.304174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.320020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.320816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.321100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.321380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.322226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:53.322536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.323968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.342486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 403/801] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:53.343428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.369674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5bfc7-cbce-469c-a065-369c53346e6e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.407206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.407910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.408204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.408476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.409255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:53.409569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.411099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.415368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.421099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 399/802] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:53.422144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.454037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bdbfc-373c-432d-8629-ef710f3d318f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.494969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.496026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.496457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.496878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.505388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.507166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.541716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e6b14-c911-4542-bc11-18d25dad25ba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.550992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.599195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.600375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.600848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.601283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1391f1de60b9465295b94c16f8a54872/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.623402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=1391f1de60b9465295b94c16f8a54872, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.626194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:53.651078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:53.652150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:09:53.653240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:53.656497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 404/803] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:53.657239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.695968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 400/804] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:53 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1391f1de60b9465295b94c16f8a54872/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:53.697276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.795614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e93e11-8cb1-43b7-9b74-82fb219a9a40 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.843514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6ba7-d5f5-4671-9706-d9608a3aa8cf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:53.858051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:53.871536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.873811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.873811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.873811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.875389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:53.875778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.877494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d24fd-c66d-40e5-bd14-a3eab6cdc4f6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.885278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 405/805] 10.222.0.93 () {56 vars in 1124 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:53.887294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098472-d582-41ef-b196-52c2569bafc9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.887585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098472-d582-41ef-b196-52c2569bafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.887866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098472-d582-41ef-b196-52c2569bafc9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.891030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 406/806] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:53.892082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098472-d582-41ef-b196-52c2569bafc9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:53.899124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:53.899854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.900143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.900475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.901332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:53.901646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:53.904544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddf5d4-194f-4ba9-973f-52d7b8751eea None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:53.927189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 401/807] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:09:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:53.928510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:53.928877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:53.929085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:53.956177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:53.957378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:53.971570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.032715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098472-d582-41ef-b196-52c2569bafc9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.041795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 402/808] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:54.041795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.087053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.087713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.087999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.088272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.089023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.090442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa234c-7f1a-4e68-9ff7-31fffc428f4c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.145604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f359e-f034-4150-9053-bdc4a33c6e03 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.180500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.181152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.181434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.181709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.189112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 407/809] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:09:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:54.190715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c26e7-ba91-4a88-a544-ec742ff3cedb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.191940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c26e7-ba91-4a88-a544-ec742ff3cedb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.192402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c26e7-ba91-4a88-a544-ec742ff3cedb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.194679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 408/810] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:54.196105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c26e7-ba91-4a88-a544-ec742ff3cedb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.198537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.271714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.297527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.299292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.341607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c26e7-ba91-4a88-a544-ec742ff3cedb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.391465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.392273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.392560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.392838 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.409752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6d776485-f633-4872-acb6-5e9195740cd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:54.419268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:54.421023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:54.422211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 409/811] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:54.422910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771715-7231-4526-b17a-5078b6487f9f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.422910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771715-7231-4526-b17a-5078b6487f9f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.423458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771715-7231-4526-b17a-5078b6487f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.443018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-78771715-7231-4526-b17a-5078b6487f9f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:54.444345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-78771715-7231-4526-b17a-5078b6487f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:54.445099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 403/812] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:09:54.445991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.497729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.529868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 410/813] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:54.531028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78771715-7231-4526-b17a-5078b6487f9f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.557828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5c082-1f47-4844-82ac-6a649703c940 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.589425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.592640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.593472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.593900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.594273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.595131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:54.595533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.596996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.606304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 404/814] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:54.612161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.667464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78771715-7231-4526-b17a-5078b6487f9f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.707256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.708304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.708759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.709179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.722849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.723494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98407d-469c-4029-8be2-907ca07700a4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.725149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.731683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:54.746025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:54.753056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 411/815] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:54.754048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.765117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.766029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.766402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.766750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] PATH_INFO: `/v3/services/network` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.767474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.769025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:54.782848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:54.783865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:54.784695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 405/816] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:54.785485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.862763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:54.879508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11147-fe3c-4fba-86bc-b373ed177dc0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.905305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c303f3-1aee-4e84-b178-17d7a94b3c6f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:54.928081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.929883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.930312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.930751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.946065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.948310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.960536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:54.961486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:54.961918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:54.962311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:54.963120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:54.963527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:54.965044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:54.972328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:54.973285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:54.974368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:54.974368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:54.974368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:54.974368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:54.982072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 412/817] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:54.983469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:54.985413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 406/818] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:54.986384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.024172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.115762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972dfcd-57e9-4104-8ccc-d4e79c989d59 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.123811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4a7b0-73f8-4fc0-ba32-b0cd17882935 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.157332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.158410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.158727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.159107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.159928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] RBAC: Adding query filter params (name=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:55.160387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.162413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.168220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 413/819] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:55.169202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.175864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.178923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.179660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.180118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.180507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.181345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:55.181750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.183422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.204101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 407/820] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:55.204101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.289552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe336-1f5f-4a7f-89f0-22ab1227c133 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.300810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.311455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767475-ae1a-4c19-a68a-81e9cb4de9a5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.329651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.330785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.331264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.332760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.344680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.346880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.350995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.351802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.352190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.352570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.353391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] RBAC: Adding query filter params (type=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:55.353836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.355335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:55.365306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:55.366318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:55.367208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:55.367661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 408/821] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:55.367661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.374834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 414/822] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:55.376106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.484446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692f8e-ea58-4534-9fbd-0e6cae8e829a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.485427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.522628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.523586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.523975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.524395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.530276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64f20-c64e-4ea4-86c4-e95255a985fb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.533561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.534849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:55.552865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:55.554105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:09:55.555260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:55.561827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 409/823] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:55.561827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.579793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.580301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.580927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.581402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.581846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.582855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.584423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.670009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 415/824] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:09:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:09:55.670009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.703934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a3c7d-04ca-4ef0-bc54-4f50c2375ae8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.790875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.791774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.792205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.793583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.794665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:55.795860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.797615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.823876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 410/825] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:55.829496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:55.890333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf119d1-a9ea-41b2-a815-e55e00de3149 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:55.917667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.958494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:55.959360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:55.959872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:55.960242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:55.961195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:55.961680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:55.966694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:55.968737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:55.998374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 416/826] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:56.005614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.026135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142c67d-e027-47d3-b452-b53c2da3b3b3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.082347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.083798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.084271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.085422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.086656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:56.087218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.088837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875dfee-f17f-4af7-8c5c-54bf8ed3684b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.096340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 411/827] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:55 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:56.141059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa69f-09c2-447e-9e24-edfbaa3f4d3c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.179692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b6f8-5b7f-4fee-8a83-eebf09b1409e None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.180507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b6f8-5b7f-4fee-8a83-eebf09b1409e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.181215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b6f8-5b7f-4fee-8a83-eebf09b1409e None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.183219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 412/828] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:56.193186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.195030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.196198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.234297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.235121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.235410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.235692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.242826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:56.244129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:56.260493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.262051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:56.288599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:56.289591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:56.290540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:56.290540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:56.290540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:56.290540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:56.297872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 417/829] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:56.299726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.367211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:56.377910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 413/830] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:56.405948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.479533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:56.508478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae61bf-04d8-4cde-9ae2-e216ad2ef087 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.526882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81924e83-b73f-4194-9956-3c1b9a4bf50c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.568111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.569030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.569414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.569699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.572043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.573102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.573588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.574097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.575351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:56.575917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.585801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.586049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:56.590159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:56.591985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:09:56.593686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:56.596513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 414/831] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:56.597632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.601564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 418/832] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:56.607639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.710526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:56.724461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5abe5-195b-4a0c-949f-c532ca7c1044 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.732583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816a3c8-bbbb-47f1-b74e-cd89dd7bbd19 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.758666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:56.762357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.763191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.763472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.763749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.771211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.771656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.771902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.777957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.777957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.778182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.778262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:56.778262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.778262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb8795-f7ff-41ef-bae9-d9013bb0680c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.787306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 415/833] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:56.788952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf7ab-ac05-4ee5-828c-7f4f18904d94 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.789543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf7ab-ac05-4ee5-828c-7f4f18904d94 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.790069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf7ab-ac05-4ee5-828c-7f4f18904d94 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:56.790369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:56.791258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:56.792161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:56.797249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 416/834] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:56.797249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddf7ab-ac05-4ee5-828c-7f4f18904d94 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.797894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 419/835] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:56.799998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:56.916124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ceed2-6c1f-44b8-9689-a29cfba61ba4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.917649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddf7ab-ac05-4ee5-828c-7f4f18904d94 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:56.957616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.958473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.958759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.959033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.959827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:56.960133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.961711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82406-4bc5-44a4-966c-96755d494620 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.969128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:56.970127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.970477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.970818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.983494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 420/836] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:56.983494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:56.983494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:56.983494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:56.990634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:56.992115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:56.998525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:56.999727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:57.002044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:57.017260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:57.018367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:57.019257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:57.019257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:57.019257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:57.019257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:57.022172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 417/837] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:57.023488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.084962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 421/838] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:57.085495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.085858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.175336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bd8a-ce96-4e23-a47b-f3f03a0e8f9c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.222460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f301-b94b-4ce8-9457-b1eae0c1505a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.229749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.230703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.230989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.231275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/4d4c712ae2b04fc68eed213213452664/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.234058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=4d4c712ae2b04fc68eed213213452664, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.235512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.263660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.264340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.264632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.264917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.265655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:57.266010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.267414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.274536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.279935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 422/839] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:57.280750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.321536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 418/840] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:09:57 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/4d4c712ae2b04fc68eed213213452664/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:09:57.321536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.421569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500942ae-1a3e-4d76-853d-b5e550005064 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.433159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.437910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e29e-84f9-4238-8a4e-17b906f4ab1c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.461297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.462152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.462440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.462721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.463559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.466870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:57.480054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:57.482375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:57.485299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 423/841] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:57.489231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.489231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.489231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.489231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.489845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.504131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.506837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:09:57.526416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:57.527435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:57.528429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 419/842] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:57.533758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.568121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.617485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e677e-9f4e-42e5-a029-bb1fecb6fc35 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.675333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.676084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.676460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.676747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.677955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:57.678429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.679789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.680773 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98addc-f133-4b00-b60f-e27c4cd8d948 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.689807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 424/843] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:57.693244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.730324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.731079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.731368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.731734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.732653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:57.733037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.734616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.754352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 420/844] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:57.761818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:57.785170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.832248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb114fd-1538-4b0a-839f-ee685cbb4607 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.863295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:57.873313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.874016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.874303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.874585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:57.875339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:57.875652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:57.877180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:57.882308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 425/845] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:09:57.887628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9eb44-2e3d-42e0-bd47-9fc970ff4b6a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:57.931951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:57.932622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:57.932902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:57.933185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.019681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.021337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.041583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:58.129856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 421/846] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:09:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:09:58.135046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.245439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35907879-affb-483a-bfb6-f22479b66737 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.282037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:58.282756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.283045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.283324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.295354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.296793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:09:58.313808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:58.314723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:58.315631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:09:58.315631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:09:58.315631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:09:58.315631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:58.315631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 426/847] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:58.321032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.426674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f27f-d1e8-4491-baf9-1f90907b72c7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.450186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:58.462813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:58.463786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.464294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.464777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.465736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:58.466889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.468464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c4252-7a7a-4240-a0aa-9d333701277e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.481432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 422/848] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:58.489845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.533026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:58.566423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326bb89-c140-45d5-83ce-c32ab76b65e1 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.567059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326bb89-c140-45d5-83ce-c32ab76b65e1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.567573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326bb89-c140-45d5-83ce-c32ab76b65e1 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.569437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 423/849] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:58.574543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.574831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.575109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.593029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:58.594251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:58.596403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ab875-d1e4-4a52-9308-91920423c2c7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.632692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:58.633507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.633824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.634115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.642510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.644103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:09:58.660306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:58.661218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:09:58.662513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:09:58.665789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 427/850] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:58.667190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 424/851] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:09:58.668078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.694469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.780821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d0c2-f4d6-4413-a755-a8e142043673 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.782863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:58.802588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05667831-bfa1-477f-aa13-fedd1277ecb8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.810189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:58.820801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:58.821813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.822307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.822865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.823823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:58.824298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.831719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.844056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:58.844056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:58.844056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:58.844056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:58.844443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:58.844759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 428/852] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:58.844955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:58.846431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:58.859252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:58.860224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:09:58.861525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:58.865707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 425/853] 10.222.0.93 () {56 vars in 1132 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:09:58.866618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:58.964825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dff0a-404e-447c-9d40-339d1d27014b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:58.980250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7642070-fd18-430a-97d3-0fdbaba8de6d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:59.007561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:59.008597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.009093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.009657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.010837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1391f1de60b9465295b94c16f8a54872) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:59.011362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:59.013785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:59.018687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:59.019356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.019641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.019925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.021083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:59.021436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:59.028428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72bdb-4fe4-421f-ad18-67330402828e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:59.028900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 429/855] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/role_assignments?user.id=1391f1de60b9465295b94c16f8a54872&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:59.033300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 426/855] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:09:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:59.034783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:59.083587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:59.171619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997fe1-9eaf-46e6-889e-b7a56c586f81 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:59.212863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:59.213933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.214443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.214727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.215449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:09:59.215763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:09:59.217211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dbbc7-0159-4ca2-8904-3399ced41727 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:09:59.222515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 430/856] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:09:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:09:59.553796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4149c2-c3c1-4c7c-9bc7-87fd2acebb73 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.554207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4149c2-c3c1-4c7c-9bc7-87fd2acebb73 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.554389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4149c2-c3c1-4c7c-9bc7-87fd2acebb73 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.556000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 427/857] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:09:59.562023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.562497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.562599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.581988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:09:59.583340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:09:59.670589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 431/858] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:09:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:09:59.694399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4149c2-c3c1-4c7c-9bc7-87fd2acebb73 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:59.739315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:09:59.820239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4149c2-c3c1-4c7c-9bc7-87fd2acebb73 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:09:59.858858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:09:59.859736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:09:59.860024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:09:59.860373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:09:59.872688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-64bba620-a75d-4b2b-bd0a-c4ad22e827c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:09:59.880936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:09:59.881976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:09:59.883268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 428/859] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:09:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:09:59.888611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:09:59.911927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:00.012263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911b705-b568-413b-9110-ad99e3d2243c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.065171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.066154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.066444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.066723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.067492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:00.067803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.069187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.080407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d03a9-588a-45e3-87a2-f7ade84486aa None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.080996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d03a9-588a-45e3-87a2-f7ade84486aa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.080996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d03a9-588a-45e3-87a2-f7ade84486aa None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.082519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 429/860] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:00.084913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 432/861] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:09:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:00.089444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.089736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.090048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.092836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.116065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:00.117287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:00.201549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 430/862] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:00.206075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:00.226907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1ad0a-be46-494c-b7f1-c2ef7f35881b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.232761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.261354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:00.271538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.272773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.273209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.274034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.288009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.290229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:00.311295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:00.312404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:00.313301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:00.313301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:00.313301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:00.313301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:00.313534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 433/863] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:00.319102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.374788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a71b-27f8-4cc0-8c5a-954f6262a255 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.422002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.422672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.422953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.423226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.423963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.425704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.451400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cf66-1fd4-4d2d-8439-b1b7d49d515a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.488153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.488866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.489246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.489537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.490305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:00.490619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.491997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.493662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:00.505569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 434/864] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:00.510825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.585502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 431/865] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:00 2022] POST /identity/v3/services => generated 251 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:00.632151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4b6f1-025b-4085-b6d1-567882bbec20 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.683285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.685141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.685638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.686218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.695794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.697881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:00.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:00.716685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 14:10:00.717656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:00.720587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 435/866] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:00.727180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.825727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:00.843811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24009d-0906-4015-aed9-6c3ee552fc9d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:00.882100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:00.882843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:00.883078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:00.883362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:00.884228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:00.884546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:00.886034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:00.907446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 432/867] 10.222.0.93 () {56 vars in 1135 bytes} [Thu Jul 14 14:10:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:00.913403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:00.978200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.029950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fd32-fe0a-4a49-a773-6ff389794deb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.067295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.067957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.068245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.068531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.141629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:01.143035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49248a-b366-4a7c-976d-c4d0b7236aa8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:01.198251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.257788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 436/868] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:01.264895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.305490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30fe4c-66ff-4e6e-9ce9-4ad9bc6f8393 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.305829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30fe4c-66ff-4e6e-9ce9-4ad9bc6f8393 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.306098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30fe4c-66ff-4e6e-9ce9-4ad9bc6f8393 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.308776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 437/869] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:01.313642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c2141-7ff2-4096-87a6-1f1cf1465363 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.313974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c2141-7ff2-4096-87a6-1f1cf1465363 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.314253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c2141-7ff2-4096-87a6-1f1cf1465363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.332827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8d7c2141-7ff2-4096-87a6-1f1cf1465363 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:01.334321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8d7c2141-7ff2-4096-87a6-1f1cf1465363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:01.375977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f69e2-2ee2-486c-bd3f-1c1855c51812 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.420364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.421615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.422202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.422619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.424598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 438/870] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:01.426731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145e476-0fa5-416a-b01f-19049ca510e2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.427126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145e476-0fa5-416a-b01f-19049ca510e2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.427507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145e476-0fa5-416a-b01f-19049ca510e2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.434212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 439/871] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:01.439748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.439748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.440128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.444256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:01.445935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:01.448039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.459952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:01.461167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:01.466413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:01.467319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:01.468217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:01.468217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:01.468217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:01.468217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:01.472475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 433/872] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:01.473285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.541047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 440/873] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:01.541968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.554327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.600853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69876dc-830c-45a7-8b28-6262ceba78fa None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.644967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.646297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.646417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.646675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.651381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3089-75a9-455a-bc93-1f41d00ee04e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.657759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:01.666642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:01.667607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:01.668448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 434/874] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:01.670097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.712764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.713679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.714145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.714573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.716259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:01.716758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:01.718899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:01.741279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 441/875] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:01.741279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.742430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.795601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4a2bf-9590-471a-a79a-d9fd2ff0b50d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.824197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:01.832394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.836602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.836602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.836602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.836602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:01.836602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:01.859258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59fdf-2460-4dbb-86f4-1814898c4ee1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:01.905404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:01.906113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:01.906399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:01.906680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:01.907434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:01.907746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:01.909118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:01.917446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 442/876] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:01.918301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:01.953987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 435/877] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:01 2022] POST /identity/v3/services => generated 269 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:01.953987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.068813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997966-3569-4c1d-8fd5-85961b5780e4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.097570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c6a2-1412-4487-aa2f-ef178e736015 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.106898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.110508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.114493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.122788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.122788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.122788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.124788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.126628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.140481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.141288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.141573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.141893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:02.147329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:02.148245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:02.149149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:02.154083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 443/878] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:02.155609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.156891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.158426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:02.203289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:02.204758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:02.206239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:02.206239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:02.206239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:02.206239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:02.208788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 436/879] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:02.216399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.288934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7bd98-2751-489c-9a9c-8bac30493c2c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.336396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.337371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.337751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.338299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.339524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:02.340195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.342273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.344770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6675ad-71ac-4f39-8433-9a9519e4c5a0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.361859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 444/880] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:02.367246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.389351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.390213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.390605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.391099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.391969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:02.392387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.393910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.412147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 437/881] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:02.413741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.433655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.449397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.501844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa88f5-a9dd-4444-8db0-344d5c6cd667 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.551856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.553052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.553440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.553872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.554489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61904c2-606e-4ef9-a55b-ff309e90102b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.554926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=4d4c712ae2b04fc68eed213213452664) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:02.555345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.558261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.567699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 445/882] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/role_assignments?user.id=4d4c712ae2b04fc68eed213213452664&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:02.600100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.600864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.601149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.601428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.611058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.612565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7dbf450e-46c7-4452-8925-858d8f04967c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:02.630981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:02.637976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:02.638932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 438/883] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:02.640247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.763373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8010-d596-47b7-94de-457375f2f852 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:02.770340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dac7-c295-4640-a55a-f967362b8a08 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.770692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dac7-c295-4640-a55a-f967362b8a08 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.770913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dac7-c295-4640-a55a-f967362b8a08 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.772585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 439/884] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:02.778784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.779079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.779361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.783227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.800994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:02.802617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:02.805338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:02.807049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:02.807343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:02.807630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:02.808505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:02.808820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:02.810600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:02.837215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 446/885] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:02.842917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.891207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 440/886] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:02.914511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:02.932737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:02.972105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540367-8ade-40a1-b6f4-7ffad5aa1ec6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.020357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.021222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.021577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.021954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.055319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ed62f-8dab-4236-ac8b-e1cfc31a8082 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.080113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.093295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.094322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.094450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.094723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.095382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.096761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:03.109030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:03.110488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:03.111341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 441/887] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:03.116683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.118408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.119984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.209879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.241451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52011c-5e39-41c4-9ca3-fbd4e60fc0d2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.242240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 447/888] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:02 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:10:03.245159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.281248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.282107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.282396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.282992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.283990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:03.284300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.286043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.296152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 442/889] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:03.299316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.337003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.373120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e771a1-f4fe-4151-9f0f-2ce11c81ae8c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.419378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.420615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.420905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.421186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.425729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1fe4-0309-47d2-b74a-69b777e35cb6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.433894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.435829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:03.460007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:03.460947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:03.461888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:03.461888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:03.461888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:03.461888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:03.466529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 448/890] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:03.467374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.467508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.468403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.468913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.469418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.470686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:03.471314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.473056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.486992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 443/891] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:03.496043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.510168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.596088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2be2-80b6-46e7-b879-87c9586fe0d1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.634710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.645576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.646279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.646572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.646854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.648056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:03.653580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.653580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.658006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc744d-67e4-4674-9803-758ae826c2e7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.670224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 449/892] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:03.672906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.716052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.717458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.718396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.719131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.721005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] RBAC: Adding query filter params (interface=public, service_id=d7af9b868f1c478bbfbba7204795de34, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:03.721831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.724317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.740853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 444/893] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/endpoints?service_id=d7af9b868f1c478bbfbba7204795de34&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:03.742821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.806140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ac6b-99be-432f-97ac-946e04531c88 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.822756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:03.857306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.858607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.858902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.859327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.861563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11e12-df7f-4b14-841f-cb90a0771183 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:03.870277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.872667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:03.893946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:03.895565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:03.896545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:03.897919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 450/894] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:03.902046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:03.912766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:03.914969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:03.915410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:03.915883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:03.917371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:03.919955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:03.928599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 445/895] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:10:03.979326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:04.062097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b428af5-ebcd-4d09-be7d-a917649cdf0d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:04.105696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:04.106442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.106728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.107006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.107794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:04.108102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:04.109587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db0bf1-0bca-468f-9ec6-9e1ca24b6754 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:04.122019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 451/896] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:04.128095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:04.245085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb68ae-9ec7-49b1-a525-a63bc46f6722 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:04.277871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:04.284290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:04.285012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.285305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.285635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.286524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] RBAC: Adding query filter params (role.id=cc954550c3f84eafa3728d648646b221, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1391f1de60b9465295b94c16f8a54872) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:04.291214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:04.291214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c80dd-52ba-43d2-aced-c0c8f7c76c3a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:04.295021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 446/897] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/role_assignments?user.id=1391f1de60b9465295b94c16f8a54872&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=cc954550c3f84eafa3728d648646b221 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:04.310876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb0b0f-085f-43ef-bfe2-32070cb36246 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.311166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb0b0f-085f-43ef-bfe2-32070cb36246 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.311489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb0b0f-085f-43ef-bfe2-32070cb36246 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.313358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 452/898] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:04.320001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.321292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.322021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.342190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:04.343711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:04.423419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 447/899] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:04.447550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb0b0f-085f-43ef-bfe2-32070cb36246 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:04.565740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb0b0f-085f-43ef-bfe2-32070cb36246 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:04.574132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:04.602899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:04.603824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.604187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.604537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.605277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:04.607330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:04.620682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:04.623139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:04.624346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 453/900] 10.222.0.93 () {56 vars in 1132 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:04.632663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:04.746746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e3dd-d6c5-42aa-8864-1d671a1a3440 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:04.775346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:04.784479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:04.785366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65beb5e-3f38-46e8-b528-9082fc233197 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:04.797937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 448/901] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:04.800854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:04.847156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfd14-8df3-4abd-b815-140486ed2a81 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.847343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfd14-8df3-4abd-b815-140486ed2a81 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.847662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfd14-8df3-4abd-b815-140486ed2a81 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.849886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 449/902] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:04.855289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.856488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.856980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.876532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:04.878373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:04.919731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a608-0607-4cf8-8795-04b754a3b6b5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:04.963136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 450/903] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:04.963993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:04.964910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:04.965987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:04.966489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:04.967422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:04.967948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:04.969530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:04.978497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 454/904] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:04.985610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:04.986655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:05.039608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:05.116556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf02a-a990-4cf0-8bfc-c15c2b3ad72a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:05.128335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd2c63-537c-47c5-be01-c39d7858f0f8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:05.154454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:05.157158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:05.158195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:05.158684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:05.159125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:05.160080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:05.161694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:05.172216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:05.172994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:05.173355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:05.173730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:05.175338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] RBAC: Adding query filter params (interface=public, service_id=d0a3eb9ae20c46cc8b7b6a7d4baee199, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:05.175940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:05.177728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:05.178491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:05.179470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:05.180363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:05.188047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 455/905] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:05.189235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:05.196702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 451/906] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:10:04 2022] GET /identity/v3/endpoints?service_id=d0a3eb9ae20c46cc8b7b6a7d4baee199&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:05.197870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:05.314322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c539a-7ac0-4c6b-bdbd-a326e80999b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:05.322613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:05.328436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30fa9-8ef4-43bd-81d6-f83488bc9091 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:05.360578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:05.361437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:05.361863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:05.362246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:05.363037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:05.364512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:05.377896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 452/907] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:05 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:10:05.380753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:05.381448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:05.381736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:05.382050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:05.382787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:05.383101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:05.384524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19b18b-1dec-425a-9ce1-3897dd43c9da None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:05.391179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 456/908] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:05.396973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:05.453710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:05.511153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f52ef-d66e-4a97-8e07-2ed3c825bc12 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:05.552643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:05.553814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:05.554338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:05.554848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:05.555864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:05.556392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:05.558068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2472fb1-f117-4d3b-92fe-5cbfb2559f1c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:05.564312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 453/909] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:06.222987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116018-2b97-453a-a16c-b6896e7bcde8 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.223515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116018-2b97-453a-a16c-b6896e7bcde8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.223515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116018-2b97-453a-a16c-b6896e7bcde8 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.225157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 457/910] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:06.231488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6243-fac1-436b-8a33-d5606517a0d5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.231990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6243-fac1-436b-8a33-d5606517a0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.231990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6243-fac1-436b-8a33-d5606517a0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.249900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1faa6243-fac1-436b-8a33-d5606517a0d5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:06.251090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1faa6243-fac1-436b-8a33-d5606517a0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:06.328393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 454/911] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:06.351399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac116018-2b97-453a-a16c-b6896e7bcde8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.459989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac116018-2b97-453a-a16c-b6896e7bcde8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:06.494697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:06.495511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.495852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.496134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.497364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:06.498754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-31ae0d29-0071-46cd-9fda-2589c7d942da None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:06.510981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:06.512034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:06.513698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 458/912] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:06.516006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d68717-8f95-4f95-8dc8-0895c6071939 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.516305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d68717-8f95-4f95-8dc8-0895c6071939 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.516585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d68717-8f95-4f95-8dc8-0895c6071939 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.521658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 455/913] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:06.524062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.524305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.524708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.526795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d68717-8f95-4f95-8dc8-0895c6071939 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.547400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:06.548603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:06.570666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:06.583108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:06.641193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 459/914] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:06.665288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d68717-8f95-4f95-8dc8-0895c6071939 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:06.668354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.704320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:06.705116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.705404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.705685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.706566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:06.707056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:06.708423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:06.714314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 456/915] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:06.722907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.812083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b8675-5e6b-40e4-83f2-dd2542ed51b8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:06.837540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489453-3bcd-4db9-940b-b83e683c1dbe None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:06.847452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:06.858088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:06.859232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.859519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.859801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.873208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:06.882321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:06.883304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:06.883837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:06.885428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:06.886345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:06.888263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:06.888878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:06.890183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:06.890725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:06.890905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 460/916] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:06.892136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.892842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:06.899855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 457/917] 10.222.0.93 () {56 vars in 1114 bytes} [Thu Jul 14 14:10:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:06.906275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:06.920434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.022014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd60b45-4c48-49fc-a449-421cb39468a3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.052729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd44f-095e-45ea-a5ff-dcd4d7ddb232 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.064816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.065966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.066400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.067031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.067662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:07.068067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.069534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.085808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 461/918] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:07.088163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.103283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.103972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.104496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.104780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.105529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.107818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.185793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 458/919] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:07.193068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.220395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89f268-1118-4ea0-8e4c-0e2ce89af34b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.268701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.269516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.269813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.270102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.282826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.284364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:07.303809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:07.304909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:07.305906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:07.305906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:07.305906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:07.305906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:07.309921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 462/920] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:07.311971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.345682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.428795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa35ec-ca32-4ad4-b966-00d09c5c8fef None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.466616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.467280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.467567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.467846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.468577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:07.468885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.470326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c131ca6-85ce-488d-b41a-6886cbdef208 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.485893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 459/921] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:07.488831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.581372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a3bd-9778-491e-b352-aab610f8d837 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.581667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a3bd-9778-491e-b352-aab610f8d837 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.581984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a3bd-9778-491e-b352-aab610f8d837 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.583693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 460/922] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:07.588912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.589199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.589480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.607392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:07.608505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:07.620660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330172a3-c983-4f89-ae3c-9e0f2d0867bc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.648689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.659907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.661256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.661744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.662253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.673039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.676036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:07.692883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:07.694335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:07.695282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:07.697458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 463/923] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:07.705182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.707107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 461/924] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:07.730419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.754266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.822078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ce5a-4e1d-4eba-b976-866a53db006c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.842395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a705-3af9-4c51-905b-c776bfdf78db None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:07.863452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.864054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.864338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.864610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.865615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:07.865963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.867424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2665823-b1a8-446e-bd53-29a5d6932657 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.882096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:07.882933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.883219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.883495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.884739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:07.886275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:07.896303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 464/925] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:07.898015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3537b64-58e3-40c3-9d65-4da193a3d47d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:07.898508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3537b64-58e3-40c3-9d65-4da193a3d47d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:07.898971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3537b64-58e3-40c3-9d65-4da193a3d47d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:07.899302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:07.900187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:07.901026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 462/926] 10.222.0.93 () {56 vars in 1132 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:07.901912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.905929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 465/927] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:07.907259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3537b64-58e3-40c3-9d65-4da193a3d47d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:07.961790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:07.997313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:08.017840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cddf9-11e0-4186-9c5f-4368a44b2dad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.021805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3537b64-58e3-40c3-9d65-4da193a3d47d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.060573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.061953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.062459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.062963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.065965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.067405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.068292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.068575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.069325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:08.069627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.071199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22295ddf-d503-4ca7-8e86-acc622c05cae None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.081101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 466/928] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:08.081758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.082283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.082741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.086956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.088509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.106800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:08.107991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:08.109565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:08.110780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:08.111701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:08.111701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:08.111701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:08.111701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:08.114174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 463/929] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:08.115327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.193825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 467/930] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:08.193825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.245358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:08.250230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdf01b-3f11-43f5-8d88-df3bb9378a3d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.282807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:08.288336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.289112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.289395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.289672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.290481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:08.290910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.292432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.305447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 464/931] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:08.305447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.316859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f96e5-d983-4fa2-a47b-1be9acbe5b08 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.355054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.355657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.355976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.356254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.357003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:08.357294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.358734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.371704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 468/932] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:08.372232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.427653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903fcca-d483-4f0d-bc9f-2c6c4710e1d2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.464824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.466199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.467361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.467361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.467550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.468926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.495634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:08.499768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16225c7-9465-4e2e-8e0a-d1c85fed4ecf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.535963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.536655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.536943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.537228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.538040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.539446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.549861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 465/933] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:08 2022] POST /identity/v3/services => generated 241 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:08.554312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.631675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:08.678525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 469/934] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:08.714921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f5fe-e53f-40a4-ba9d-3e02080b18ac None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.771697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.771967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.772093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.772448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.781461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.783365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:08.801226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:08.802432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:08.803550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:08.805448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 466/935] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:08.812248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:08.925816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc37f-afca-4e2a-8da2-b102c7161a95 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:08.963953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:08.964629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:08.964914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:08.965276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:08.966320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:08.966636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:08.968087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4894e6-ad67-4c6b-885f-acae31583836 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:08.981575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 470/936] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:08.987421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:09.122843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb9cb8-a9de-440d-8c6d-e39667d0d3c0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:09.133067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:09.160300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:09.161938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:09.162443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:09.162940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1391f1de60b9465295b94c16f8a54872/roles/cc954550c3f84eafa3728d648646b221` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:09.166422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=1391f1de60b9465295b94c16f8a54872, role_id=cc954550c3f84eafa3728d648646b221)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:09.168039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295b9cf-806d-4ae9-96f0-9206cd0657e2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:09.239328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 467/937] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:10:08 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/1391f1de60b9465295b94c16f8a54872/roles/cc954550c3f84eafa3728d648646b221 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:10:09.502312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c2b1-95cb-4292-bbb6-5cf047c9db12 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:09.502825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c2b1-95cb-4292-bbb6-5cf047c9db12 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:09.502825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c2b1-95cb-4292-bbb6-5cf047c9db12 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:09.504633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 471/938] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:09.511951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:09.512887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:09.513397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:09.534508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:09.536084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:09.613871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 468/939] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:09.636073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c2b1-95cb-4292-bbb6-5cf047c9db12 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:09.656332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:09.757271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c2b1-95cb-4292-bbb6-5cf047c9db12 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:09.795777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:09.796820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:09.797270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:09.797698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:09.798613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:09.800272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:09.801867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING keystone.server.flask.common [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 14:10:09.851567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.sql.core [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 14:10:09.851567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 14:10:09.851567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [parameters: {'id': '261e4b91682f41efbd95b3df89d8c328', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 14:10:09.851567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80327) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 261e4b91682f41efbd95b3df89d8c328.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 261e4b91682f41efbd95b3df89d8c328. Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:10:09.869897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 14:10:09.870953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application yield resource Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise self.value Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application session.commit() Jul 14 14:10:09.872155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.flush() Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application compat.raise_( Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 14:10:09.873290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 14:10:09.874518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:10:09.875379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [parameters: {'id': '261e4b91682f41efbd95b3df89d8c328', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:09.876275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:09.877076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 261e4b91682f41efbd95b3df89d8c328. Jul 14 14:10:09.878110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:09.880603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 472/940] 10.222.0.93 () {60 vars in 1140 bytes} [Thu Jul 14 14:10:09 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:09.887490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:09.995776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:10.000257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb844793-49b3-48d3-99f2-0b121fe47fcd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:10.040961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:10.041882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.042275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.042658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:10.050431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:10.052179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ebc228e1-6e87-42d6-8218-21703cb35179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:10.067631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:10.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:10.069859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:10.075035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 469/941] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:10.083135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:10.176302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:10.221914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5837a-e306-44dd-a2c5-94551f8223c8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:10.257859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:10.258909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.259401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.259892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:10.260914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:10.261455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:10.264030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab74420-a7b3-4dce-b673-0fa2250dd58f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:10.289907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 473/942] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:10.823184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970a62-9e9a-4050-a454-4739d559c70f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.823747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970a62-9e9a-4050-a454-4739d559c70f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.823747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970a62-9e9a-4050-a454-4739d559c70f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:10.825392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 470/943] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:10.831135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01c14-b6bd-4a81-ae39-605079bfed9f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.831623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01c14-b6bd-4a81-ae39-605079bfed9f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.831755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01c14-b6bd-4a81-ae39-605079bfed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:10.850025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3ee01c14-b6bd-4a81-ae39-605079bfed9f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:10.851573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3ee01c14-b6bd-4a81-ae39-605079bfed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:10.925936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 474/944] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:10.948954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53970a62-9e9a-4050-a454-4739d559c70f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:10.983537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5ce6-7b3a-44ee-b133-1afd0ba6f1ce None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.983828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5ce6-7b3a-44ee-b133-1afd0ba6f1ce None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.984116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5ce6-7b3a-44ee-b133-1afd0ba6f1ce None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:10.985786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 475/945] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:10.991379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:10.991676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:10.991960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.010104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:11.011932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:11.072824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53970a62-9e9a-4050-a454-4739d559c70f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.101095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.126445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 476/946] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:11.139809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.140984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.141593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.142121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.143181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.144823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.149843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:11.160762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:11.162350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:11.163442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:11.167294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 471/947] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:11.168879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.195919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.266567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7767c76-7844-416d-8dbf-7482a6b8c561 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.279027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c003b1-7a59-45bb-be2c-c0a718caffd3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.311764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.313026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.313543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.314104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.314976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.316826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.317576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.318393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.318686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.318972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.319694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:11.320009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.321610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.327651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 472/948] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:11.332447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:11.333462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:11.334356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:11.338567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 477/949] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:11.338984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.339424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.403292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.453790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.455049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb3d4-7cbe-4808-bdba-2e3479e30dd4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.476150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6384-c1bc-4c63-9c53-518937fdd879 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.496502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.497202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.497526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.497934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.498698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] RBAC: Adding query filter params (name=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:11.499018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.500462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.510199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 478/950] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:11.513497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.530162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.532100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.532392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.532830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.533566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:11.533908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.535299 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046e9b0-5030-4d6b-b965-153699575730 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.540679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 473/951] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:11.542012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe42ae-46e1-4873-80f1-eb66dee6e410 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.542309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe42ae-46e1-4873-80f1-eb66dee6e410 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.542589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe42ae-46e1-4873-80f1-eb66dee6e410 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.544082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 474/952] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:11.547188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe42ae-46e1-4873-80f1-eb66dee6e410 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.633155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a376e8-8c26-4947-97e4-cd0f464b81de None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.653436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.663621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe42ae-46e1-4873-80f1-eb66dee6e410 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.673658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.674365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.674654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.674935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.675654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:11.675965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.677335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80d0ae-0717-4a0a-bc04-3131a99383e0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.689909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 479/953] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:11.689909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.689909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.689909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.706951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:11.708398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:11.715241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.715924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.716211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.716489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.717301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] RBAC: Adding query filter params (interface=public, service_id=05ca3f18294e4710ab1be5ac8de3b65c, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:11.717618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.719218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.726629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 475/954] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/endpoints?service_id=05ca3f18294e4710ab1be5ac8de3b65c&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:11.732040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.756448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.796360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 480/955] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:11.822780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:11.881313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7388a2-8721-492a-9369-4e26401a7e68 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.926898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.927784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:11.928077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:11.928362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:11.929322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:11.931494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:11.940487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 476/956] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:10:11.954550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9564b-ec1a-4199-83cc-c421c1623d2e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:11.972426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:11.999044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:11.999955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.000248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.000530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.014590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:12.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:12.025178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 481/957] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:12.032670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.140362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:12.163625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe938b41-ca11-4b15-8886-b2d9711a6dae None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.206171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.207713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.208184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.208616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.209547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:12.210048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.211598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:12.222275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 477/958] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:12.228061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.336594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd789-0657-45de-bb53-36d1988a309f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.373474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.374615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.375075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.375502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.387528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.389187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4b9ea12e-f53a-4939-a3ba-b045d4c16ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:12.407629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:12.408638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:12.409549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:12.409549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:12.409549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:12.409549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.411395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 482/959] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:12.417068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.458870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491052d-5676-4909-88dc-edd7bfc0bff6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.459137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491052d-5676-4909-88dc-edd7bfc0bff6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.459375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491052d-5676-4909-88dc-edd7bfc0bff6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.461041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 483/960] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:12.466142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.466437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.466722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.488165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:12.489712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:12.510084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:12.541420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f452c71-adb1-4f3a-8b16-3b0771270735 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.568603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 484/961] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:12.580977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.581650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.581967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.582251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.582984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:12.583319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.584698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:12.593539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.594454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:12.597921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 478/962] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:12.604021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.717986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e704-d06c-41c9-ba0d-6ff719d2a236 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.718224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e30ac-df70-459f-92b6-92e25fa2e0c1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.760909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.761128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.762443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.762651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.762923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.763027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.763203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.763442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.764275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.766517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:12.771960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.773533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:12.790732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:12.791965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:12.794991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 479/963] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:12.798474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.811589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.sql.core [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 14:10:12.811589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 14:10:12.811589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [parameters: {'user_id': '8e206169eb4a4f60b0c6e9c28c5a92cf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 14:10:12.811589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80327) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 14 14:10:12.822448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:10:12.823584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application yield resource Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise self.value Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application session.commit() Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 14:10:12.824528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.flush() Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application compat.raise_( Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 14:10:12.825417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 14:10:12.826384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 14:10:12.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [parameters: {'user_id': '8e206169eb4a4f60b0c6e9c28c5a92cf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:12.829079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 14:10:12.830380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:12.834463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 485/964] 10.222.0.93 () {60 vars in 1131 bytes} [Thu Jul 14 14:10:12 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:12.839320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:12.865564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:12.883632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:12.939649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4268-aafe-4a8d-ae2b-0655cd520baa None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.970342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de212-6b1d-4fd3-a844-ac99a987da94 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:12.980170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:12.981793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:12.982224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:12.982743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:12.983726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:12.984183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:12.985806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.009160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 480/965] 10.222.0.93 () {56 vars in 1149 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:13.010519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.017614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.018500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.018987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.019484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.030346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.032141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:13.051362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:13.052353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 14:10:13.053292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:13.058906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 486/966] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:13.058906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.131975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075cf04-8e62-41f5-9dc8-0b78017f5757 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:13.156369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:13.160336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:13.180354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.181521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.182279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.182839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.185664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37ccf-dbda-46ea-b421-d0f62235af11 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:13.224915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.225511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.225944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.226231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.227161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:13.227477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.228923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.266913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.268679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.273026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 487/967] 10.222.0.93 () {56 vars in 1101 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:13.388489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 481/968] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:13.395816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.506603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a73d7-4c8b-4467-a751-94cf0c7ce916 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:13.552788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.553603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.553925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.554202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.556501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:13.569244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.571405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:13.590873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:13.591829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:13.592939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:13.592939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:13.592939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:13.592939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:13.592939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 488/969] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:13.599570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.620595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:13.701424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4b66b-0f3b-43c0-b9a2-e21889d1f87c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:13.736224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.736889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.737172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.737931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.738372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:13.738675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.740037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.756715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 482/970] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:13.758062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.865516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454cb9-ae3c-4f56-a042-825affc31771 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:13.900918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:13.901710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.902039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.902866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.909790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:13.911186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6726fb9a-2a5a-420d-a08d-aa69441369f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:13.924895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:13.926582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:13.928184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:13.929353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 489/971] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:13.932024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:13.966461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a18dc6-f0a6-45d8-ac11-b31a5b44f304 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.966756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a18dc6-f0a6-45d8-ac11-b31a5b44f304 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.967044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a18dc6-f0a6-45d8-ac11-b31a5b44f304 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.968556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 490/972] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:10:13.973721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:13.974420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:13.974420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:13.993126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:13.994709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:14.026022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.045194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.052271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bc9cb-95dc-4155-9a62-8ec5578e80b0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.082104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 491/973] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:14.100469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.101147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.101435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.101735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.102514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:14.102827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.105095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:14.106913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.128248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 483/974] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:14.129516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:14.229957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfd57-d03a-423b-9fca-4ad4968f4f74 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.249919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead0c9-0a73-4ec7-b486-fc3b7b4f809b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.266380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.267042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.267332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.267612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.268346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.269723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.284930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.285510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.285830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.286113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.286948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] RBAC: Adding query filter params (role.id=cc954550c3f84eafa3728d648646b221, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=1391f1de60b9465295b94c16f8a54872) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:14.287260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.288690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf035ec5-20c8-4aaa-9252-4d5b166a854f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.295302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 484/975] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:14 2022] GET /identity/v3/role_assignments?user.id=1391f1de60b9465295b94c16f8a54872&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=cc954550c3f84eafa3728d648646b221 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:14.296580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe06b80-795b-4d9e-ae00-12a9010b26e7 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.296913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe06b80-795b-4d9e-ae00-12a9010b26e7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.297144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe06b80-795b-4d9e-ae00-12a9010b26e7 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.298398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 485/976] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:14.305447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.305723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.306042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.311880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.325495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:14.326761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:14.375466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 492/977] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:14 2022] POST /identity/v3/services => generated 242 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:14.417030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 486/978] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:14.444166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:14.494070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.570644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6e07-3d83-4fa7-a435-c72297ab5d57 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.608022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.608960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.609148 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.609492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.610143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.611511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:14.624585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:14.626530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:14.627751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 493/979] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:14.631959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:14.743448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ca27-045c-40f3-9f01-0e24835b3e43 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.775523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.784414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.785086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.785373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.785921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.786644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:14.786951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.788339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.798708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 487/980] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:14.800235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:14.905886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:14.909925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a62e7-9c91-48fd-adf4-98afff44c704 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:14.948627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:14.949942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:14.950230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:14.950504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:14.951222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:14.951532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:14.952910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0c56-34d3-4835-97bb-1992daf9a3c2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:14.963115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 494/981] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:14.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:15.092572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0062c9-ce83-41aa-b2cf-c5eba62a6cbf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:15.136849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:15.137522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:15.137842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:15.138129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:15.138883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:15.140270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3a323-5965-4a3f-9bc0-dd363efe6b0f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:15.227190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 488/982] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:16.383303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e071-73f0-42b6-826c-b40e032fbd06 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.383837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e071-73f0-42b6-826c-b40e032fbd06 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.383837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e071-73f0-42b6-826c-b40e032fbd06 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.385335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 495/983] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:16.390741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.391207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.391285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.409406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:16.410630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:16.461339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:16.475226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67b3bd-4bd3-4a27-a170-2a7d716220a2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.475527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67b3bd-4bd3-4a27-a170-2a7d716220a2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.475812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67b3bd-4bd3-4a27-a170-2a7d716220a2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.485075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 496/984] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:10:16.485075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.485075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.485075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.492349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 489/985] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:16.499582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:16.500828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:16.515138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.575792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 497/986] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:16.600630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.636016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b41c46-7277-4cef-80ef-561a15baf3ee None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:16.678306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:16.679105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.679382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.679653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.680309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:16.681687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:16.693862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:16.695051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:16.701369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 490/987] 10.222.0.93 () {56 vars in 1105 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:16.702241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.710894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:16.720710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8bcb-49ed-48a0-af5b-191ffa611650 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:16.748645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:16.757150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:16.758055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.758341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.758625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.759332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:16.760701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:16.773922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:16.774958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:16.775978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:16.781115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 498/988] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:16.782001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.820450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640a1a-cec3-4d05-9ca1-4a49ba4d23d6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:16.860086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:16.860764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.861051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.861326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.862076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] RBAC: Adding query filter params (name=image) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:16.862738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:16.863787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:16.869484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 491/989] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:16.875315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.891210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1177-4fea-412f-8541-9cd5fb50b97d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:16.930665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:16.931350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:16.931637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:16.931956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:16.932628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:16.932939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:16.934891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:16.940664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 499/990] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:16.946366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:16.973359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.001810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191909b-3baf-46a3-bc6b-b572c4617e96 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.003710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.045161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.046223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.046731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.047165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.048032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:17.048476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.050045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.055565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 492/991] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:17.082722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65982d-d18d-484a-91cd-b5a72ff5416d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.125951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.126780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.127134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.127490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.128469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:17.128808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.130361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33b03b-ec21-4345-832f-a05a29cbf9a8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.137301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 500/992] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:17.146859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:17.271658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9c5cd-cd95-46fb-9e0c-4a90a665c2a4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.312566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.313538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.313538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.313864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.314856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] RBAC: Adding query filter params (interface=public, service_id=54a861c4eb80495687cb3fa7d6a5da24, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:17.315173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.315421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40e1d5-2c60-458a-a5b5-1b74007a5c14 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.316182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40e1d5-2c60-458a-a5b5-1b74007a5c14 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.316182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40e1d5-2c60-458a-a5b5-1b74007a5c14 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.316548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.318124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 501/993] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:17.322651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 493/994] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:10:17 2022] GET /identity/v3/endpoints?service_id=54a861c4eb80495687cb3fa7d6a5da24&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:17.323275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.323840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.324314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.327920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:17.344622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:17.346027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:17.360365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.435649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 502/995] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:17.446095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc28eaa-931d-4ee4-a946-34ec6723a1fb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.458529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:17.476797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.487355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.490772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.490772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.490772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.490772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.490772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.510068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 494/996] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:17 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:10:17.580499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbfdb5-923d-46cd-b4d0-2802f3cfc637 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.621930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.622869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.623163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.623477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.624285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.625903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:17.639577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:17.640658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:17.646855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 503/997] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:17.647712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:17.759123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.766271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8655c-8571-496e-8732-c529ce35299d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.810051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.810735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.811024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.811311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.812065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:17.812371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.813758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.824356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 495/998] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:17.825490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:17.913549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:17.941404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bf858-7d77-411d-ad6f-a46c05706d16 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:17.987347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:17.988309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:17.988799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:17.989266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:17.990269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:17.990738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:17.992276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:17.998978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 504/999] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:18.004916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:18.129172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82caf15-23d1-46f2-b296-8822e059de2d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:18.175405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:18.176350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:18.176848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:18.177282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:18.178419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] RBAC: Adding query filter params (interface=admin, service_id=05ca3f18294e4710ab1be5ac8de3b65c, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:18.178911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:18.180557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e36a2e-4820-4ad4-b411-6ae82774ee76 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:18.187293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 496/1000] 10.222.0.93 () {56 vars in 1249 bytes} [Thu Jul 14 14:10:18 2022] GET /identity/v3/endpoints?service_id=05ca3f18294e4710ab1be5ac8de3b65c&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:18.193182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:18.324585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9937674-f4c9-433c-8209-9764a4ee0298 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:18.365652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:18.366375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:18.366671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:18.367006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:18.367715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:18.369099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644a9fa-e606-4611-b427-92c2a5f25d1d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:18.377564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 505/1001] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:18 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:10:19.283913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e0530-d207-4d9f-a854-9d0b22d39117 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:19.284441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e0530-d207-4d9f-a854-9d0b22d39117 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:19.284549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e0530-d207-4d9f-a854-9d0b22d39117 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:19.285874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 497/1002] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:19.291582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196b07-e2ad-44f5-a27b-4aa03ad932bf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:19.292098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196b07-e2ad-44f5-a27b-4aa03ad932bf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:19.292180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196b07-e2ad-44f5-a27b-4aa03ad932bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:19.310009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f1196b07-e2ad-44f5-a27b-4aa03ad932bf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:19.311265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f1196b07-e2ad-44f5-a27b-4aa03ad932bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:19.313350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:19.389891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 506/1003] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:19.422050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e0530-d207-4d9f-a854-9d0b22d39117 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:19.449347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:19.554090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e0530-d207-4d9f-a854-9d0b22d39117 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:19.598003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:19.598635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:19.598923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:19.599195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:19.599932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:19.601305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:19.650142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea36f5-b95f-4051-9f03-d58f692bd1df None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:19.650442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea36f5-b95f-4051-9f03-d58f692bd1df None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:19.650972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea36f5-b95f-4051-9f03-d58f692bd1df None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:19.652967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 507/1004] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:19.660267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:19.660553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:19.661593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:19.680101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:19.680101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:19.708900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 498/1005] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:19 2022] POST /identity/v3/services => generated 242 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:19.799818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 508/1006] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:19 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:19.827005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:19.936090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:19.961340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7f7e-341d-4c05-a66a-e4f1cd4dc88c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:20.015353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:20.016478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.017001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.017486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.018395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:20.020110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:20.033829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:20.034989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 14:10:20.035991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.037875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 499/1007] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:20.042862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:20.141052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:20.207152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bcadf-f8b5-481d-a789-e720c3f96711 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:20.244828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:20.246117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.246409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.246693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.247414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:20.247727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:20.249111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:20.259230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 509/1008] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:20.260571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:20.379811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861f954-84f9-4d06-8099-8009802c18e4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:20.421742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:20.422440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.422721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.422997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.423716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:20.424020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:20.425393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b8438-2cfc-44f1-80ed-9f9fcad096d4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:20.436662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 500/1009] 10.222.0.93 () {56 vars in 1112 bytes} [Thu Jul 14 14:10:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:20.442329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:20.525027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0ab50-20b7-4b4f-a3ea-e89a0c4c04e5 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.525389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0ab50-20b7-4b4f-a3ea-e89a0c4c04e5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.525632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0ab50-20b7-4b4f-a3ea-e89a0c4c04e5 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.527806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 501/1010] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:20.532815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.533102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.533386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.554276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:20.555488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:20.559387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c4e0-2d09-40c1-8161-252d3484ef2e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:20.560908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:20.592417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:20.601385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:20.602286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.602702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.603142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.604060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:20.605968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:20.633985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 502/1011] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:20 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:20.658585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:20.733557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 510/1012] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:20.777186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb0f1-6f4f-4ca0-9987-9e28b7882623 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:20.825548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:20.826419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:20.826706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:20.826981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:20.827662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:20.829089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:20.832578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:20.843095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:20.850143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:20.851000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 503/1013] 10.222.0.93 () {56 vars in 1126 bytes} [Thu Jul 14 14:10:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:20.853261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:20.987373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710d5f-8886-47bc-b5e6-9da2fd94c3d1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:21.029030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:21.029634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:21.029971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:21.030248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:21.030969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:21.031277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:21.032640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:21.039897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 511/1014] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:21.048508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:21.168745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5223-b934-487f-81de-3de7f91e358a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:21.207597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:21.208551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:21.208837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:21.209115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:21.209864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:21.210266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:21.212670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd30912-6ab6-4fbc-a7d5-6abcd867dff2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:21.221392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 504/1015] 10.222.0.93 () {56 vars in 1122 bytes} [Thu Jul 14 14:10:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:21.229045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:21.285866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:21.347226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f48ca-5a89-4255-a9fa-fcd7cad46125 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:21.387616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:21.388539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:21.388972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:21.389461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:21.390427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:21.392063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e248804-7810-451e-b292-e17d5ff1ddd2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:21.527351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 512/1016] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:21.860888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e29-60a5-4036-88e9-c95525f0c775 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:21.861419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e29-60a5-4036-88e9-c95525f0c775 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:21.861534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e29-60a5-4036-88e9-c95525f0c775 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:21.862882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 505/1017] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:21.868258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:21.868558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:21.868843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:21.885655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:21.887439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:21.972753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 513/1018] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:21.996340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967e29-60a5-4036-88e9-c95525f0c775 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:22.072934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:22.124082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967e29-60a5-4036-88e9-c95525f0c775 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:22.166871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:22.167697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.167987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.168275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] PATH_INFO: `/v3/services/image` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.168936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:22.170541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:22.182537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:22.190534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:22.191397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 506/1019] 10.222.0.93 () {56 vars in 1105 bytes} [Thu Jul 14 14:10:21 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:22.192284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:22.226899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:22.299142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf41085-e4d5-494d-97fe-8ce77ca8df4e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:22.336975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:22.337489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.339544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.340903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.341651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] RBAC: Adding query filter params (name=image) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:22.342010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:22.343408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:22.355156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 514/1020] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:22.355371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:22.466146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd420d-fb6a-4131-93f9-a924c1f0e950 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:22.494386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:22.505148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:22.505855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.506146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.506428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.507144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] RBAC: Adding query filter params (type=image) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:22.507455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:22.508837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:22.519614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 507/1021] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:22.520807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:22.624145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:22.631732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fabbf-486c-4b9f-bbd8-fd49df0e446b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:22.682424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:22.683696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.683980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.684260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.685086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] RBAC: Adding query filter params (interface=public, service_id=642e96c1ed724842ad00456d3944856e, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:22.685401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:22.687064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1056d-b002-48c4-a54e-81e505cf7772 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:22.698046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 515/1022] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:10:22 2022] GET /identity/v3/endpoints?service_id=642e96c1ed724842ad00456d3944856e&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:22.699755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:22.818132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb8c3-24f5-4cef-88ef-d597cebc3a6c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:22.860754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:22.861434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.862047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.862047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.862698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:22.864082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb64d3d-85e2-4b7c-9d3e-9a0aa2927920 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:22.872003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 508/1023] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:10:22 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:10:22.961851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e54d41-8d6d-4e62-ba35-35728bef90ef None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.962184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e54d41-8d6d-4e62-ba35-35728bef90ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.962424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e54d41-8d6d-4e62-ba35-35728bef90ef None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.964033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 516/1024] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:22.969573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:22.969906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:22.970198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:22.988854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:22.990271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:23.053804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:23.087044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 509/1025] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:23.110143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e54d41-8d6d-4e62-ba35-35728bef90ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.235867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e54d41-8d6d-4e62-ba35-35728bef90ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:23.274241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:23.275067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.275351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.275626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.290104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:23.299051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:23.300155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:23.301003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 517/1026] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:23.308830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.416260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99bb7a-9140-4b47-8487-ffa14d3eaafd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:23.457466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:23.458184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.458470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.458744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.459499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:23.459804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:23.461196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1046cb-65ad-4794-b6ac-5efe9e59ef1c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:23.476501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 510/1027] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:23.478270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.508040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:23.592116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588dea1-49cc-4782-9789-c9e8245b18f4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:23.628072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:23.628911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.629142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.629420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.640524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:23.641995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:23.659799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:23.660766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:23.669182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:23.669182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:23.669182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:23.669182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:23.669182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 518/1028] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:23.669495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdffd1-bc9d-4b38-bb02-4f5ace52cceb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.669495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdffd1-bc9d-4b38-bb02-4f5ace52cceb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.669495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdffd1-bc9d-4b38-bb02-4f5ace52cceb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.669681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.671128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 511/1029] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:23.676251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.676759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.677220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.698440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:23.700796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:23.750231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:23.760477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:23.794138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393e3c-55d8-4708-b889-8431be125b71 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:23.795338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 512/1030] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:23.818910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.835804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:23.836477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:23.836769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:23.837050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:23.837806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:23.838123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:23.840027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:23.856041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 519/1031] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:23.861973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:23.958364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb9e62-1014-46ec-a15d-f1348ec1a746 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:23.979097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaefb90-3fce-46af-88f3-a0703c36ae98 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.006769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.007927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.007927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.008204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.009332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.011397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.013064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 513/1032] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:24.019490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:24.028850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.029926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.030379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.031166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.034487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:24.042635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.044168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:24.061231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:24.066066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:24.067549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:24.068119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 520/1033] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:10:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:24.068773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:24.086749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:24.144681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2646ce6-52b6-41f9-b2f5-f5d16bd72617 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.186713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.187390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.187682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.187967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.188683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.190728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.195682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e027-2857-4837-8972-f2eb39b7dd65 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.231533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.232459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.232741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.233017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.233875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:24.234185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.235776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.254583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 521/1034] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:10:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:24.260505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:24.301928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 514/1035] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:24 2022] POST /identity/v3/projects => generated 311 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:10:24.340591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:24.410470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb8ee8-b44a-49f2-9d1e-6568a8c7339e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.450438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.451105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.451394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.451672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.526667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.528145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.580665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:24.631299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 522/1036] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:10:24.636825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:24.741423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:24.745929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688ecf-9611-4627-9e7a-770efad019ab None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.783055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.784113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.784409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.784684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.795724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.797177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:24.816592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:24.817644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:24.819687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:24.819687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:24.819687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:24.819687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:24.819687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 515/1037] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:24.824042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:24.928576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fc801-0730-4c46-9f7c-99921b40465f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:24.965290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:24.965987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:24.966277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:24.966558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:24.967294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:24.967607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:24.968987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a95f-f128-4f0c-a59f-364124e26a53 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:24.981447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 523/1038] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:24.987093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.012093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c4d9d-0a5a-49eb-9fe9-a1bb588ac40d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.012407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c4d9d-0a5a-49eb-9fe9-a1bb588ac40d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.012675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c4d9d-0a5a-49eb-9fe9-a1bb588ac40d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.014316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 524/1039] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:10:25.020069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.020384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.020618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.037821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:25.039027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:25.047562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.109742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450c3d-ea3a-48c3-9962-52329c6ee50e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.117821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 525/1040] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:25.139348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.150241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.151070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.151973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.152248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.159552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.160988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:25.175928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:25.176998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:25.177983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 516/1041] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:25.182998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.196797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.266296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd3894-aa9b-4023-a617-94859105ffff None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.290367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a5f92-b3c2-4e33-9c7f-88d26b88780e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.305314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.306151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.306478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.306716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] PATH_INFO: `/v3/services/image` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.307884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.309753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:25.331706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:25.332903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:25.333759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 526/1042] 10.222.0.93 () {56 vars in 1105 bytes} [Thu Jul 14 14:10:25 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:25.339673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.341614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.342330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.342611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.342886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.343678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:25.343985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.345429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.351094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.358138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 517/1043] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:25.364096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.441953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.466866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d339e-a041-4ba5-b964-ee4f1b6f94dc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.480511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce152-3e6d-49f5-8b11-536ce1ed2ee8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.510326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.511807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.512097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.512383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.513106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:25.513940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.515012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.521555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 527/1044] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:25.526899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.529385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.530142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.530436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.530716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.531558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] RBAC: Adding query filter params (role.id=cc954550c3f84eafa3728d648646b221, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:25.531870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.533362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.545903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 518/1045] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:25 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=cc954550c3f84eafa3728d648646b221 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:25.614142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.649611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caceb1f-1d0e-4ba4-83bb-ea689e791bb2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.688143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.688827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.689372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.690061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.690534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:25.690826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.692211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff450a-4783-45aa-bb1d-bb99a2f40bb2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.699027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 528/1046] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:25.704273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:25.828079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1feac8-a1fd-4505-a1fd-ed5fc2964aba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:25.863002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:25.863670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:25.864051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:25.864319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:25.865069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:25.866456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8bcfd-d128-4b96-a7d5-a7e52dab3546 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:25.869864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:25.997819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 519/1047] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:10:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:26.436902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4962a5-da0a-4997-beb1-6ef912cde0ac None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:26.437289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4962a5-da0a-4997-beb1-6ef912cde0ac None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:26.437469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4962a5-da0a-4997-beb1-6ef912cde0ac None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:26.439290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 529/1048] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:26.444528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:26.444829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:26.445111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:26.461898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:26.463093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:26.540630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 520/1049] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:26.566758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4962a5-da0a-4997-beb1-6ef912cde0ac None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:26.692919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4962a5-da0a-4997-beb1-6ef912cde0ac None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:26.734808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:26.735610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:26.735894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:26.736171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:26.737350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:26.739117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cd5b7-7fdf-4df9-a723-2861737b8b62 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:26.740981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 530/1050] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:26.746560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:26.850819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e755e-6568-46eb-8dee-6bd15f612b98 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:26.868585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:26.886613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:26.887630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:26.888144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:26.888630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:26.889567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:26.891283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0ee62-2ed7-44b0-bcaa-d6724258befc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:26.976902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 521/1051] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:10:26 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:27.614109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce30f7-fcf3-46fd-b80b-5001431502a5 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:27.614480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce30f7-fcf3-46fd-b80b-5001431502a5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:27.614678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce30f7-fcf3-46fd-b80b-5001431502a5 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:27.616195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 531/1052] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:27.621262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:27.621725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:27.621839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:27.639955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:27.641148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:27.719346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 522/1053] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:27.742718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce30f7-fcf3-46fd-b80b-5001431502a5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:27.745869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:27.856914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce30f7-fcf3-46fd-b80b-5001431502a5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:27.891623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:27.892358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:27.892642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:27.892919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:27.904964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:27.913231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:27.914362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:27.915230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:27.918122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 532/1054] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:27.920559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.023410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c58bb-0d4d-48da-9bc1-44fa7f11d5b3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.024971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.062643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.062795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.063060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.063352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.064588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:28.065372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.067468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54b279-a05b-42f3-bd24-19404e59c021 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.081089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 523/1055] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:28.084472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.141064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe3c8-36ed-4700-8b8b-662231b1b932 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.141363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe3c8-36ed-4700-8b8b-662231b1b932 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.141711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe3c8-36ed-4700-8b8b-662231b1b932 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.143202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 524/1056] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:28.148192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.148543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.148827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.156021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.167597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:28.168860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:28.202604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603b88-f745-4c51-acb6-56c1e80bb960 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.244278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.245142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.245400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.245660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.253281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 525/1057] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:28.257261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.258745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.274750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:28.279530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:28.280412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:28.281326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:28.281326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:28.281326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:28.281326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:28.281326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 533/1058] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:28.286996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.382571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.390920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8809-9306-4fd7-b6f1-2feb316a098e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.411729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea95f3-4768-4edd-bf6b-2f9b7cc3c4f8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.427729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.431909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.432616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.432906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.433187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.449669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.450440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.450730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.451032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.451777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:28.452112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.453474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.467524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 534/1059] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:28.473167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.518292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.519767 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.589291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57838-6eb6-4b56-b35c-5ef4a1693cea None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.630396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.640188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 526/1060] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:28.642935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.643742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.644026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.647465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.648804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.656286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.659023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:28.686213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:28.687409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:28.688604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:28.689442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 535/1061] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:28.695104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.733592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.758878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f6723-a60f-4ad2-b6e9-eb70aff230ab None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.802922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.803729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.803997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.804271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.808956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43811-20b7-4553-a789-9101b6d199ec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.815802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.817273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:28.838068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:28.840142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:28.841209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:28.841209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:28.841209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:28.841209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:28.841209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 527/1062] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:28.846233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.850710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:28.851397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:28.851686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:28.851970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:28.852828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:28.853147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:28.854666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:28.875193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 536/1063] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:28.881297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:28.903996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.963221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bd56-c410-4d0a-b36a-bcd10c8a5b9f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:28.985041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:28.994053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001453ff-26bc-49e3-8f1d-2bce306c2e2d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.002240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.002901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.003187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.003464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.004479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:29.004479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.005908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.026421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 528/1064] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:29.032248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.033258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.033527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.034074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.046168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.047670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ee0d2421-1084-4187-9439-e8cae5da0f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:29.068843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:29.069861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:29.076275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:29.076275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:29.076275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:29.076275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:29.076275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 537/1065] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:29.076872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.131748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0788-a645-4da7-a8fe-288ba3528e50 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.132039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0788-a645-4da7-a8fe-288ba3528e50 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.132330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0788-a645-4da7-a8fe-288ba3528e50 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.134032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 538/1066] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:10:29.140060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.140361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.140636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.159943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:29.161190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:29.199513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebdde-3c8b-4fb0-863d-399f2ed0a3ef None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.209910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.239645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.240311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.240902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.241803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.242909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:29.243225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.245125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.245719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 539/1067] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:29.261865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 529/1068] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:29.267038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.271693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.355372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.380853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f6e90-eef3-487c-b933-7a23f32c349f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.386630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0dbd-bbdb-4b48-b362-ca6cbb9504a0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.415419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.416218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.417029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.417309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.421681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.422501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.422783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.423055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.424214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.425663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.434843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:29.440392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:29.441451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:29.442407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 540/1069] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:29.442905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:29.443748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:29.449787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 530/1070] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:29.450721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.450808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.478399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.553177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc19b2-02f6-4368-8d17-f4bb445e7c91 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.554447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade196e-693e-40c2-88c9-16054080a581 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.588241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.588915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.589201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.589483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.590272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:29.590371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.590585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.591035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.591325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.591603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.591955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.592394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:29.592707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.593490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.594158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.601562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 541/1071] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:29.607979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.609513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 531/1072] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:29.616279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.708408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.722002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e9c7-db10-4ced-af7b-701be36a102b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.724303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b5d5-c75e-4cd8-b56d-530368ce496a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.758864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.759745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.760056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.760336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] PATH_INFO: `/v3/users/484a72a179084cc19dbd6c96d0657609` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.775442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] RBAC: Authorizing `identity:get_user(user_id=484a72a179084cc19dbd6c96d0657609)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.777601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.779350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 542/1073] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/users/484a72a179084cc19dbd6c96d0657609 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:29.781530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.782464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.782754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.783039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/cc954550c3f84eafa3728d648646b221` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:29.784987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:29.785508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=47007139f45e4e55b5a9fa2d2901ef9c, role_id=cc954550c3f84eafa3728d648646b221)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:29.786912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:29.851510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:29.863250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 532/1074] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:10:29 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/47007139f45e4e55b5a9fa2d2901ef9c/roles/cc954550c3f84eafa3728d648646b221 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:10:29.904670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289247f-8239-425e-ae86-670505f63a4c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:29.950769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:29.951444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:29.951755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:29.952017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:30.030730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:30.032202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:30.093600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:30.141140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 543/1075] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:10:30.146868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:30.276407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609d97f-81c9-43d6-a9d5-cbc25c51d579 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:30.324180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:30.325067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:30.325351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:30.325631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:30.326974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:30.328420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a88405-bfef-4824-97b8-634cc1af362d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:30.330196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 533/1076] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:30 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:30.336230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:30.476485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1ff04-fb6a-4685-8c29-6f68dd337225 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:30.517745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:30.518433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:30.518718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:30.518996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:30.519836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=e78d7270c41b4052b9ccbbce4f876da9, user.id=484a72a179084cc19dbd6c96d0657609) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:30.520150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:30.523069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de32a3-cb72-4d4d-a832-79c87da9039d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:30.528528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 544/1077] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:30 2022] GET /identity/v3/role_assignments?user.id=484a72a179084cc19dbd6c96d0657609&scope.project.id=e78d7270c41b4052b9ccbbce4f876da9&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:31.978215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423c0f-f02c-49d8-a7df-80bc0d5a1136 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:31.978835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423c0f-f02c-49d8-a7df-80bc0d5a1136 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:31.978835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423c0f-f02c-49d8-a7df-80bc0d5a1136 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:31.980392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 534/1078] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:31.985612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:31.986033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:31.986292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.003409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:32.004588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:32.018513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:32.076913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 545/1079] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:32.101793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47423c0f-f02c-49d8-a7df-80bc0d5a1136 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:32.117804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:32.206627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47423c0f-f02c-49d8-a7df-80bc0d5a1136 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:32.240968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:32.241806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.242095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.242367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.253283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:32.261916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:32.263078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:32.268985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 535/1080] 10.222.0.93 () {56 vars in 1120 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:32.269951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:32.372334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9b08d-afdf-4bd4-92b4-05887a9d2476 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:32.411513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:32.412183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.412468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.412745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.413494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:32.413839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:32.415317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:32.424369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 546/1081] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:32.430145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:32.508736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:32.533500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000cea2-c49e-4451-a3c9-215ed6bd32e5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:32.569639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:32.570493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.570779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.571058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.582352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:32.583814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:32.601157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:32.602229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:32.604223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:32.604223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:32.604223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:32.604223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:32.604223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 536/1082] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:32.608659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:32.622387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:32.713075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff06f-d007-48c4-b3c7-8ae8791e70eb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:32.748482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:32.749154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.749445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.749729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.750542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:32.750814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:32.752252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac159f30-4341-4f42-815c-ba053c7154df None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:32.807376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 547/1083] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:32.808431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:32.818699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbadb35-195d-4e87-87c3-f7fbdee38f33 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.818989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbadb35-195d-4e87-87c3-f7fbdee38f33 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.819267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbadb35-195d-4e87-87c3-f7fbdee38f33 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.820756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 548/1084] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:10:32.827801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:32.828096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:32.828387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:32.846102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:32.847325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:32.968032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:32.970963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057b14a-7612-48d2-9878-35135ba16755 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:32.972769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 549/1085] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:32.996220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.003997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.014410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.015205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.015485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.015757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.024469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.026059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:10:33.041621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:33.042874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 14:10:33.044010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:33.044987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 537/1086] 10.222.0.93 () {56 vars in 1093 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:33.049387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.109032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4ce87-1cca-415d-858a-d74066090002 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.148450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.149250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.149531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.149837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.157285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb6c-6ae3-44e8-a4fc-f6b0b2bb89b5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.160476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:33.168637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:33.169602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:33.170539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 550/1087] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:33.177166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.195754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.196429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.196711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.196993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.197899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:33.198210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.199644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.217423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 538/1088] 10.222.0.93 () {56 vars in 1133 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:33.223271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.242751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.269809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.284546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95fb3-e6c0-4390-9bf3-efd2ff0e0420 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.327406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.327406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.327406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.327406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.327838 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:33.327838 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.328951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.332969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad77bb4-1c10-4df9-8fe2-2226fade1be9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.338053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 551/1089] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:33.343863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.368317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.368976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.369825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.370394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.448673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.450238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.465544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a6f0-ad57-4428-9deb-cd4a76c891a7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.491873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.501929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.504785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] PATH_INFO: `/v3/users/484a72a179084cc19dbd6c96d0657609` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=484a72a179084cc19dbd6c96d0657609)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.521358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 552/1090] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/users/484a72a179084cc19dbd6c96d0657609 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:33.521833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.559578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 539/1091] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:10:33.565855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.623627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b7569-944b-4f6a-96a4-5e4eee260dd5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.657586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.658474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.658762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.659050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.660234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.661820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.663681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 553/1092] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:33.665613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d65fa-79df-4836-ac14-8f3be90cc5e3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.670050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.703032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.703868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.704153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.704433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.717355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.718851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.721752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.734550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:10:33.747032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:33.748033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:33.749213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:10:33.749213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:10:33.749213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:10:33.749213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:33.749213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 540/1093] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:10:33.757017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:33.780959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386858de-3f9a-4701-947c-56d046ab2b9b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.831510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.832410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.832698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.832982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9/users/484a72a179084cc19dbd6c96d0657609/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.835816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78d7270c41b4052b9ccbbce4f876da9, user_id=484a72a179084cc19dbd6c96d0657609, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.837315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.887912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d753ba-d634-410f-abd3-79877b90fde8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:33.918328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 554/1094] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:10:33 2022] PUT /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9/users/484a72a179084cc19dbd6c96d0657609/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:10:33.927490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:33.928188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:33.928472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:33.928749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:33.929509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:33.929846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:33.931287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:33.950428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 541/1095] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:10:33.951770 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:34.069074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785dba1-cc1a-4237-82e7-45ecce715313 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:34.087782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:34.105213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:34.106046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:34.106382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:34.106623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:34.113380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:34.114867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:10:34.128932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:34.130733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:10:34.132360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 555/1096] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:34.140418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:34.203948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:34.256139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222361-1dad-4adc-b380-fa3d23091157 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:34.289762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:34.290440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:34.290729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:34.291035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:34.291834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:34.292147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:34.293569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c90f69-7b21-4a43-91d3-c86bf7677eaf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:34.305512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 542/1097] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:10:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:34.312144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:34.412216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d04-3f97-453b-b1b9-7f066bc4fdf3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:34.446051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:34.446701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:34.446983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:34.447834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:34.448717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] RBAC: Adding query filter params (role.id=cc954550c3f84eafa3728d648646b221, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=47007139f45e4e55b5a9fa2d2901ef9c) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:34.449028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:34.450482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3d2e6-fc7c-42d9-893c-f8a485624c16 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:34.456361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 556/1098] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:34 2022] GET /identity/v3/role_assignments?user.id=47007139f45e4e55b5a9fa2d2901ef9c&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=cc954550c3f84eafa3728d648646b221 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:36.003703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6644-b102-4521-9735-13a871d184fe None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.004357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6644-b102-4521-9735-13a871d184fe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.004357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6644-b102-4521-9735-13a871d184fe None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.005617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 543/1099] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:36.011076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.011552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.011649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.028803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:36.030033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:36.066379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:36.101564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 557/1100] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:36.125182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f6644-b102-4521-9735-13a871d184fe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:36.228881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f6644-b102-4521-9735-13a871d184fe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:36.248817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:36.269006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:36.269715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.270024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.270304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.281995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:36.289734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:36.291604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:36.293293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 544/1101] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:36.297061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:36.399063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbb7ce-a7c6-4a0c-b53f-2e074daed27c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:36.434072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:36.434747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.435036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.435316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.436069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:36.436386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:36.437754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:36.446329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 558/1102] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:36.452919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:36.558438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f440012-e246-4814-b024-2c146387fee0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:36.592539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:36.593406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.593685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.593984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] PATH_INFO: `/v3/users/484a72a179084cc19dbd6c96d0657609` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.595171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] RBAC: Authorizing `identity:get_user(user_id=484a72a179084cc19dbd6c96d0657609)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:36.596597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:36.598255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 545/1103] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:36 2022] GET /identity/v3/users/484a72a179084cc19dbd6c96d0657609 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:36.604302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:36.640504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:36.706029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc4826-1342-44bb-b633-c784a300b060 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:36.740028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:36.740677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:36.740960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:36.741235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:36.809008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:36.810402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:36.844457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:36.908992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 559/1104] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:36.914845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:36.966226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:37.018943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26a4d-f931-44ad-bf09-003bfffff2ac None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:37.052724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:37.053697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:37.054102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:37.054396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:37.055944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:37.057428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854c90e-a4d2-4e04-8d0a-47204a6ad3e4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:37.059247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 546/1105] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:36 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:37.065531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:37.165383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631be01-cfd1-4465-ab61-6aa669b95617 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:37.200776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:37.201437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:37.201728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:37.202046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:37.202881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=e78d7270c41b4052b9ccbbce4f876da9, user.id=484a72a179084cc19dbd6c96d0657609) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:37.203196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:37.204628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2a3e-abeb-4bb7-a116-c3b74dce72f0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:37.206598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:37.211249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 560/1106] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:37 2022] GET /identity/v3/role_assignments?user.id=484a72a179084cc19dbd6c96d0657609&scope.project.id=e78d7270c41b4052b9ccbbce4f876da9&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:39.285860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99519b0a-a307-4f0d-80b8-b353ba95b235 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:39.286451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99519b0a-a307-4f0d-80b8-b353ba95b235 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:39.286451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99519b0a-a307-4f0d-80b8-b353ba95b235 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:39.287827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 547/1107] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:39.293937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:39.294447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:39.294559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:39.312684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:39.315021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:39.396422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 561/1108] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:39.423073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99519b0a-a307-4f0d-80b8-b353ba95b235 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:39.559031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99519b0a-a307-4f0d-80b8-b353ba95b235 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:39.573122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:39.594796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:39.595600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:39.595881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:39.596159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:39.598036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:39.599613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05246ebf-429a-454f-93c7-890928851d2c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:39.606123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 548/1109] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:39.607429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:39.727281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac924b-b3d3-4595-8350-291d47166356 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:39.755073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:39.766770 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:39.767472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:39.767760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:39.768045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:39.768899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:39.770396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892de75-96eb-485b-a06b-2d960a0c8dda None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:39.846670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 562/1110] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:10:39 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:41.989194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731186a6-888b-4d92-a10d-d207daecaaf8 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:41.989739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731186a6-888b-4d92-a10d-d207daecaaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:41.989739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731186a6-888b-4d92-a10d-d207daecaaf8 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:41.991924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 549/1111] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:41.998133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:41.998701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:41.998701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:42.016587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:42.017837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:42.088537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 563/1112] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:42.112626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731186a6-888b-4d92-a10d-d207daecaaf8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:42.220567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731186a6-888b-4d92-a10d-d207daecaaf8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:42.255702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:42.256466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:42.256747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:42.257023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] PATH_INFO: `/v3/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:42.258507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] RBAC: Authorizing `identity:get_role(role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:42.259888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:42.261264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 550/1113] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:42 2022] GET /identity/v3/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:42.267599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:42.370379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ea1a3-993f-4c1e-8eb5-0b32e8e6406c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:42.384760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:42.406251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:42.407120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:42.407401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:42.407677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] PATH_INFO: `/v3/users/4435e68c82a949cab9d0aa7461d770d5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:42.422219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] RBAC: Authorizing `identity:get_user(user_id=4435e68c82a949cab9d0aa7461d770d5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:42.423657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:42.425337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 564/1114] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:42 2022] GET /identity/v3/users/4435e68c82a949cab9d0aa7461d770d5 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:42.431318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:42.481793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:42.539022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728f34-596e-4429-bde3-fbf9915e9db8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:42.573884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:42.574530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:42.574815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:42.575091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:42.642300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:42.643800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:42.748673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:42.801446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 551/1115] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:42.807755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:42.914285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a1e2-06f0-489c-b28b-109440af388b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:42.948261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:42.949115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:42.949395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:42.949666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:42.950887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:42.952313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747442c-5ae5-41aa-9762-2db3d3750765 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:42.954058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 565/1116] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:42 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:42.960369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:43.060283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9bde5-335c-46ee-81fd-20a2f733bee8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:43.096144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:43.096804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:43.097092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:43.097372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:43.098242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=e78d7270c41b4052b9ccbbce4f876da9, user.id=4435e68c82a949cab9d0aa7461d770d5) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:43.098557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:43.099994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba257d74-3cc8-4b7c-a91c-f2b6b81924b7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:43.105450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 552/1117] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:42 2022] GET /identity/v3/role_assignments?user.id=4435e68c82a949cab9d0aa7461d770d5&scope.project.id=e78d7270c41b4052b9ccbbce4f876da9&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:45.113396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c258-2261-4c1a-bebb-a7dab2aea353 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.113980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c258-2261-4c1a-bebb-a7dab2aea353 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.113980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c258-2261-4c1a-bebb-a7dab2aea353 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.115347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 566/1118] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:45.120546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.120920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.121152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.139036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:45.140438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:45.148001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:45.210022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 553/1119] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:45.233211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c258-2261-4c1a-bebb-a7dab2aea353 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:45.277363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:45.338126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c258-2261-4c1a-bebb-a7dab2aea353 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:45.373753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:45.374650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.374936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.375216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] PATH_INFO: `/v3/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.376767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] RBAC: Authorizing `identity:get_role(role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:45.378323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:45.379739 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 567/1120] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:45 2022] GET /identity/v3/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:45.386175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:45.488708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27258f8e-f046-4c98-b07a-e9b205a370aa None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:45.523698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:45.524597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.524886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.525167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] PATH_INFO: `/v3/users/4435e68c82a949cab9d0aa7461d770d5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.526418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] RBAC: Authorizing `identity:get_user(user_id=4435e68c82a949cab9d0aa7461d770d5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:45.527857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:45.529516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 554/1121] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:45 2022] GET /identity/v3/users/4435e68c82a949cab9d0aa7461d770d5 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:45.535459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:45.641833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8e4ef-5ef9-4d1d-84b8-5a6ddf316a89 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:45.651095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:45.675453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:45.676318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.676608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.676888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.678099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:45.679533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919ef2e-0313-4823-86c6-ce2bd10b2a9f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:45.681787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 568/1122] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:45 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:45.688121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:45.701058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:45.790538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa112-2e2d-4937-8c94-509d8ac45d24 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:45.825099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:45.826021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:45.826318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:45.826598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9/users/4435e68c82a949cab9d0aa7461d770d5/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:45.828782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78d7270c41b4052b9ccbbce4f876da9, user_id=4435e68c82a949cab9d0aa7461d770d5, role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:45.830262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd8925-e368-425b-9d18-1843e3d19e06 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:45.913843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 555/1123] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:10:45 2022] PUT /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9/users/4435e68c82a949cab9d0aa7461d770d5/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:10:47.938449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140fff-e683-4bbb-a4fd-33df2438d4a4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:47.939016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140fff-e683-4bbb-a4fd-33df2438d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:47.939016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140fff-e683-4bbb-a4fd-33df2438d4a4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:47.940373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 569/1124] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:47.945729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:47.946809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:47.946913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:47.965559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:47.966801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:48.029682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:48.034749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 556/1125] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:48.059500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08140fff-e683-4bbb-a4fd-33df2438d4a4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:48.161831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08140fff-e683-4bbb-a4fd-33df2438d4a4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:48.197141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:48.197940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:48.198225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:48.198502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] PATH_INFO: `/v3/roles/3de12fa346dc48bcba0e5747639c52f0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:48.199970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] RBAC: Authorizing `identity:get_role(role_id=3de12fa346dc48bcba0e5747639c52f0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:48.201334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:48.202738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 570/1126] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:48 2022] GET /identity/v3/roles/3de12fa346dc48bcba0e5747639c52f0 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:48.208581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:48.317080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71bbf6-48dd-45de-b011-3446ab190a5d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:48.353258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:48.354164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:48.354451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:48.354731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] PATH_INFO: `/v3/users/4435e68c82a949cab9d0aa7461d770d5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:48.355943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] RBAC: Authorizing `identity:get_user(user_id=4435e68c82a949cab9d0aa7461d770d5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:48.357391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:48.359078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 557/1127] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:48 2022] GET /identity/v3/users/4435e68c82a949cab9d0aa7461d770d5 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:48.364907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:48.412501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:48.467302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a04b-2c46-408e-a430-22c6e84d5920 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:48.503059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:48.503701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:48.503984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:48.504259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:48.578004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:48.579489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:48.625748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:48.681997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 571/1128] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:10:48.688338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:48.754851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:48.788511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee7134-e58b-4851-bb0f-52b51513264f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:48.822798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:48.823651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:48.823937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:48.824215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] PATH_INFO: `/v3/projects/e78d7270c41b4052b9ccbbce4f876da9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:48.825437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] RBAC: Authorizing `identity:get_project(project_id=e78d7270c41b4052b9ccbbce4f876da9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:48.826912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661021be-943a-42b5-a114-efbcf6a738af None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:48.828637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 558/1129] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:48 2022] GET /identity/v3/projects/e78d7270c41b4052b9ccbbce4f876da9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:48.834920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:48.933600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a44628-ce55-4a95-9168-6aa2a18b09bd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:48.974612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:48.975291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:48.975537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:48.975812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:48.976640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] RBAC: Adding query filter params (role.id=3de12fa346dc48bcba0e5747639c52f0, scope.project.id=e78d7270c41b4052b9ccbbce4f876da9, user.id=4435e68c82a949cab9d0aa7461d770d5) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:48.976953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:48.978504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba47a4-e8dc-4600-b75e-d5e6500d936c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:48.984374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 572/1130] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:48 2022] GET /identity/v3/role_assignments?user.id=4435e68c82a949cab9d0aa7461d770d5&scope.project.id=e78d7270c41b4052b9ccbbce4f876da9&role.id=3de12fa346dc48bcba0e5747639c52f0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:51.056271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1eb0-db6b-4f50-8124-885ebab0a83a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:51.056810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1eb0-db6b-4f50-8124-885ebab0a83a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:51.056810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1eb0-db6b-4f50-8124-885ebab0a83a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:51.058205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 559/1131] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:51.063727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:51.064178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:51.064272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:51.080625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:51.081900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:51.083852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:51.147754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 573/1132] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:10:51.171952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b1eb0-db6b-4f50-8124-885ebab0a83a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:51.280665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b1eb0-db6b-4f50-8124-885ebab0a83a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:51.316229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:51.316928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:51.317208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:51.317484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:51.318690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:51.320113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c00fa-b50e-4495-9b56-3595b3047f85 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:51.321746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 560/1133] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:51.327726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:51.447108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94616e78-4bcf-44d3-85a2-a139faf28eba None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:51.497177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:51.497999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:51.498190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:51.498465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:51.499213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:51.500703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397e662-8c98-44b0-b0e4-16e1ad1a801e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:51.509981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:51.625632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 574/1134] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:10:51 2022] POST /identity/v3/projects => generated 311 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:53.702889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b717b-bbc8-4995-8481-672a8b617e0c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:53.703462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b717b-bbc8-4995-8481-672a8b617e0c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:53.703462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b717b-bbc8-4995-8481-672a8b617e0c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:53.704781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 561/1135] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:53.710173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:53.710634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:53.710759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:53.728712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:53.729951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:53.796096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 575/1136] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:53.820777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b717b-bbc8-4995-8481-672a8b617e0c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:53.824594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:53.927864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b717b-bbc8-4995-8481-672a8b617e0c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:53.962499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:53.963204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:53.963490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:53.963765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:53.964950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:53.966576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3778e1-7b1a-4c39-b212-99e70dfd297e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:53.968212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 562/1137] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:10:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:53.974084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:54.074450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842ad0a-e962-4498-ad35-a37a2f6ad6b9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:54.095733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:54.109730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:54.110429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:54.111796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:54.111796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:54.111796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:54.113112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172f533-98cf-40f1-9a01-42bcf7971c8a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:54.205354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 576/1138] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:10:53 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:10:56.314943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d140f4-8ded-41f6-9c39-e16c37affe72 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:56.315473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d140f4-8ded-41f6-9c39-e16c37affe72 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:56.315473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d140f4-8ded-41f6-9c39-e16c37affe72 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:56.316770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 563/1139] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:56.321847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:56.322293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:56.322397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:56.339341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:56.340514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:56.404608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 577/1140] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:56.428610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d140f4-8ded-41f6-9c39-e16c37affe72 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:56.498540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:56.527859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d140f4-8ded-41f6-9c39-e16c37affe72 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:56.561691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:56.562874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:56.563156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:56.563433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:56.573446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:56.580919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:56.582076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:56.583117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 564/1141] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:56.588224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:56.693869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c8e3b-3de3-4970-97cc-cbc787d161dc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:56.703483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:56.735624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:56.736316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:56.736765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:56.737490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:56.738348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:56.738725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:56.740174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:56.748839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 578/1142] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:56.755133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:56.874658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d2adc-9e8d-42ad-9d28-6e68d6b04a8d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:56.909390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:56.914826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:56.915690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:56.916023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:56.916301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] PATH_INFO: `/v3/users/3070f5187beb473e99fc9a4ad683ddf9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:56.930929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=3070f5187beb473e99fc9a4ad683ddf9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:56.932478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:56.934294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 565/1143] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:10:56 2022] GET /identity/v3/users/3070f5187beb473e99fc9a4ad683ddf9 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:56.940587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:57.045668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b6f1-006a-4f6c-8261-2c54a6755cc5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:57.080655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:57.081317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:57.081605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:57.081929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:57.117462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:57.151834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:57.153193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:57.255549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 579/1144] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:10:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:10:57.262051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:57.361410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000b493-f8d9-4fac-9ede-dc2c7d757fb2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:57.395260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:57.396110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:57.396403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:57.396681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] PATH_INFO: `/v3/projects/5caaf9a41d1048df8330b014c196a424` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:57.397890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] RBAC: Authorizing `identity:get_project(project_id=5caaf9a41d1048df8330b014c196a424)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:57.400028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653317a-3230-459b-ae1a-a0d9020a08bf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:57.401725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 566/1145] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:10:57 2022] GET /identity/v3/projects/5caaf9a41d1048df8330b014c196a424 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:57.407929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:57.473149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:57.505314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9e8a6-e894-42d8-ba6d-777079b6dcb7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:57.540102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:57.540771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:57.541054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:57.541337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:57.542213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=5caaf9a41d1048df8330b014c196a424, user.id=3070f5187beb473e99fc9a4ad683ddf9) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:57.542527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:57.543965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3bba-f30a-434f-8b86-cd42738debcf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:57.550479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 580/1146] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:10:57 2022] GET /identity/v3/role_assignments?user.id=3070f5187beb473e99fc9a4ad683ddf9&scope.project.id=5caaf9a41d1048df8330b014c196a424&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:10:59.558475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac392bca-3414-4fcf-99a9-a286a9eb7469 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:59.559006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac392bca-3414-4fcf-99a9-a286a9eb7469 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:59.559096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac392bca-3414-4fcf-99a9-a286a9eb7469 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:59.560386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 567/1147] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:10:59.565973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:59.566348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:59.566532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:59.583562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:10:59.584753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:10:59.659029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 581/1148] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:10:59.681753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac392bca-3414-4fcf-99a9-a286a9eb7469 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:59.743232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:59.785307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac392bca-3414-4fcf-99a9-a286a9eb7469 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:59.819192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:59.819192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:59.819192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:59.819192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:59.833578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:10:59.841132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:10:59.842250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:10:59.843444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 568/1149] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:10:59.848793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:10:59.911803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:10:59.951571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a2fa-8e44-4579-abdd-7d112ef8df5e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:10:59.985232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:10:59.985911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:10:59.986196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:10:59.986468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:10:59.987212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:10:59.987515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:10:59.988897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:10:59.997596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 582/1150] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:00.003495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:00.134335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf01dd-3cd9-48e8-a54d-641bdb009414 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:00.160303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:00.177644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:00.178546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:00.178833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:00.179114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] PATH_INFO: `/v3/users/3070f5187beb473e99fc9a4ad683ddf9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:00.180931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] RBAC: Authorizing `identity:get_user(user_id=3070f5187beb473e99fc9a4ad683ddf9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:00.182423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:00.184191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 569/1151] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:11:00 2022] GET /identity/v3/users/3070f5187beb473e99fc9a4ad683ddf9 => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:00.190547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:00.318209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c5b7-24b8-4b3e-a6a7-0bfc0e1c49d5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:00.345273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:00.356379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:00.357246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:00.357534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:00.357844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] PATH_INFO: `/v3/projects/5caaf9a41d1048df8330b014c196a424` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:00.359052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] RBAC: Authorizing `identity:get_project(project_id=5caaf9a41d1048df8330b014c196a424)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:00.360495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2131-2e20-49a2-8aad-50136462554c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:00.362257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 583/1152] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:11:00 2022] GET /identity/v3/projects/5caaf9a41d1048df8330b014c196a424 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:00.369102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:00.470338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fed0-d480-4a5b-bdde-6bd53ac426ff None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:00.505754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:00.506664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:00.506951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:00.507226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] PATH_INFO: `/v3/projects/5caaf9a41d1048df8330b014c196a424/users/3070f5187beb473e99fc9a4ad683ddf9/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:00.509216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5caaf9a41d1048df8330b014c196a424, user_id=3070f5187beb473e99fc9a4ad683ddf9, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:00.510684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b95ca-f323-4c5b-9efa-1b924c9474e9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:00.632965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 570/1153] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:11:00 2022] PUT /identity/v3/projects/5caaf9a41d1048df8330b014c196a424/users/3070f5187beb473e99fc9a4ad683ddf9/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:11:02.684388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf610-a676-4d25-b10f-1148cc695582 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:02.684931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf610-a676-4d25-b10f-1148cc695582 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:02.685026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf610-a676-4d25-b10f-1148cc695582 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:02.686394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 584/1154] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:02.691807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:02.692237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:02.692374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:02.709700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:02.710924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:02.752664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:02.777185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 571/1155] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:02.800369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acf610-a676-4d25-b10f-1148cc695582 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:02.907798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acf610-a676-4d25-b10f-1148cc695582 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:02.944579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:02.945937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:02.946223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:02.946497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:02.956993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:11:02.964411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:11:02.965533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:11:02.966604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 585/1156] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:11:02.972199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:03.069088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9315-19f7-4a06-994c-4ba44ba1223a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:03.103517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:03.104187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:03.104475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:03.104757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:03.105508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:03.105852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:03.107227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:03.115355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 572/1157] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:03.121851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:03.174885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:03.227454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7395c-328b-4927-9f16-7c8ea6078695 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:03.262547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:03.263420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:03.263705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:03.263983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] PATH_INFO: `/v3/users/3070f5187beb473e99fc9a4ad683ddf9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:03.265214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] RBAC: Authorizing `identity:get_user(user_id=3070f5187beb473e99fc9a4ad683ddf9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:03.266677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:03.268336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 586/1158] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:11:03 2022] GET /identity/v3/users/3070f5187beb473e99fc9a4ad683ddf9 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:03.274290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:03.316669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:03.378238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb7753-0919-4418-b6cb-86505a124360 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:03.412739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:03.413383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:03.414279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:03.414566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:03.481514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:03.482761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:03.521954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:03.598553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 573/1159] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:11:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:11:03.605457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:03.712013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b31ff-e280-48ac-9a5e-a4e60331b260 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:03.733168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:03.746837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:03.747706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:03.748961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:03.748961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] PATH_INFO: `/v3/projects/5caaf9a41d1048df8330b014c196a424` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:03.749590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] RBAC: Authorizing `identity:get_project(project_id=5caaf9a41d1048df8330b014c196a424)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:03.751074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb013-2bd2-46a1-b94f-9b139f10c6ca None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:03.752783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 587/1160] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:11:03 2022] GET /identity/v3/projects/5caaf9a41d1048df8330b014c196a424 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:03.760759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:03.863321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c39b7-8c8e-44ce-a3d2-26aa7f209e29 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:03.898906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:03.899820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:03.900219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:03.900498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:03.901327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=5caaf9a41d1048df8330b014c196a424, user.id=3070f5187beb473e99fc9a4ad683ddf9) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:03.901641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:03.903147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847e71f-eeaf-43bf-a5e9-963a39652c88 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:03.913164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 574/1161] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:11:03 2022] GET /identity/v3/role_assignments?user.id=3070f5187beb473e99fc9a4ad683ddf9&scope.project.id=5caaf9a41d1048df8330b014c196a424&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:06.077888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00de344-9f86-4401-bec1-98d8817d60fd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:06.078440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00de344-9f86-4401-bec1-98d8817d60fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:06.078440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00de344-9f86-4401-bec1-98d8817d60fd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:06.079817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 588/1162] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:06.085113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:06.085569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:06.085669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:06.090800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:06.103171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:06.104371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:06.173197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 575/1163] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:06.196432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00de344-9f86-4401-bec1-98d8817d60fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:06.298093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00de344-9f86-4401-bec1-98d8817d60fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:06.331833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:06.332406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:06.332695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:06.332977 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:06.388637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:06.401261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:06.406656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:06.499414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 589/1164] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:11:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:11:06.505486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:06.603932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712add62-8617-4416-9c7f-4ec50b9661f7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:06.635952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:06.641012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:06.654010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:06.654010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:06.654010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:06.654010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:06.654010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e8fed1b5-9df8-41fa-ae86-d0fce2302c66 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:11:06.666130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:11:06.667458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:11:06.668575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 576/1165] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:11:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:11:06.673367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:06.763230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:06.775263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77309faa-7d99-4054-80ba-f5f0980bdc92 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:06.814413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:06.815081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:06.815373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:06.815655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:06.816382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:06.816693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:06.818093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7982a22-8a3c-4b88-abe3-6b76ff3762ba None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:06.826363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 590/1166] 10.222.0.93 () {56 vars in 1115 bytes} [Thu Jul 14 14:11:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:08.849807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb0562-d681-448b-a3cf-2714b61986f6 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:08.850379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb0562-d681-448b-a3cf-2714b61986f6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:08.850379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb0562-d681-448b-a3cf-2714b61986f6 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:08.851821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 577/1167] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:08.857373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473f06-88e9-461e-8802-ac1918666bad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:08.857863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473f06-88e9-461e-8802-ac1918666bad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:08.857976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473f06-88e9-461e-8802-ac1918666bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:08.875232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f4473f06-88e9-461e-8802-ac1918666bad None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:08.876452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f4473f06-88e9-461e-8802-ac1918666bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:08.945365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 591/1168] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:08.968562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb0562-d681-448b-a3cf-2714b61986f6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:09.079199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb0562-d681-448b-a3cf-2714b61986f6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:09.113641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:09.114255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:09.114545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:09.114826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:09.115590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:09.117633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607ef2c-32ef-44b4-9a51-7f20279fbb3e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:09.176292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 578/1169] 10.222.0.93 () {60 vars in 1138 bytes} [Thu Jul 14 14:11:08 2022] POST /identity/v3/domains => generated 267 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:11:11.231178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e64ed7-ef0f-4e4e-91cd-9f6f4560898a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:11.231737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e64ed7-ef0f-4e4e-91cd-9f6f4560898a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:11.231737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e64ed7-ef0f-4e4e-91cd-9f6f4560898a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:11.233214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 592/1170] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:11.238707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:11.239167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:11.239260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:11.255607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:11.256825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:11.302334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:11.321859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 579/1171] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:11.346436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e64ed7-ef0f-4e4e-91cd-9f6f4560898a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:11.427427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:11.446443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e64ed7-ef0f-4e4e-91cd-9f6f4560898a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:11.479928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:11.480854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:11.481145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:11.481423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] PATH_INFO: `/v3/domains/a79c07f2a18944fb9470a0f126849841` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:11.492120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a79c07f2a18944fb9470a0f126849841)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:11.493550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d8f3-2ab9-4a18-8ca1-5c7fa1a81264 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:11.495235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 593/1172] 10.222.0.93 () {56 vars in 1183 bytes} [Thu Jul 14 14:11:11 2022] GET /identity/v3/domains/a79c07f2a18944fb9470a0f126849841 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:11.501249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:11.597952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d66066-6c5e-4fe2-9849-4d07e2f58838 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:11.630443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:11.631108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:11.631398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:11.632214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:11.632925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:11.634396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da6d1-9d0c-41ee-afc0-c3b536bdf028 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:11.753231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 580/1173] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:11:11 2022] POST /identity/v3/projects => generated 361 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:11:13.827569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8974a8-52d5-442b-b2f0-5f98df4de465 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:13.828137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8974a8-52d5-442b-b2f0-5f98df4de465 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:13.828137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8974a8-52d5-442b-b2f0-5f98df4de465 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:13.829506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 594/1174] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:13.835330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:13.835786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:13.835881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:13.852633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:13.853861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:13.879614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:13.932115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 581/1175] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:13.958116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8974a8-52d5-442b-b2f0-5f98df4de465 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:14.064717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8974a8-52d5-442b-b2f0-5f98df4de465 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:14.099681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:14.100466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:14.100753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:14.101035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] PATH_INFO: `/v3/domains/a79c07f2a18944fb9470a0f126849841` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:14.102254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a79c07f2a18944fb9470a0f126849841)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:14.103684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23b04f-5e89-40cc-8420-0bc647e2dd15 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:14.105326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 595/1176] 10.222.0.93 () {56 vars in 1183 bytes} [Thu Jul 14 14:11:13 2022] GET /identity/v3/domains/a79c07f2a18944fb9470a0f126849841 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:14.111123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:14.212312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff334-08ed-40e3-b767-f4476c0b672f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:14.247462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:14.248123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:14.248411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:14.248693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:14.249412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:14.250879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e22d90-d7ac-4903-b498-f2913b91380f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:14.334404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 582/1177] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:11:14 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:11:16.421407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01434c46-4bfa-48ce-b5ab-0009a2356878 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:16.422079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01434c46-4bfa-48ce-b5ab-0009a2356878 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:16.422079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01434c46-4bfa-48ce-b5ab-0009a2356878 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:16.423286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 596/1178] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:16.428564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:16.428991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:16.429123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:16.446067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:16.447270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:16.461663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:16.513117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 583/1179] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:16.544959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01434c46-4bfa-48ce-b5ab-0009a2356878 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:16.557623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:16.646603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01434c46-4bfa-48ce-b5ab-0009a2356878 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:16.683287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:16.683995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:16.684280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:16.684558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:16.695472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:11:16.703285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:11:16.704423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:11:16.705486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 597/1180] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:11:16.710715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:16.814808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fa4c6-cc1a-4d8d-b83f-8eb1e36711c7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:16.848011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:16.848671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:16.848952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:16.849227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:16.850526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:16.850829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:16.852197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:16.860734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 584/1181] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:16.867028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:16.936478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:16.969698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48128e-3dfc-42b8-9bc0-9200af7b0f0d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:17.005600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:17.007190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:17.007476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:17.007752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] PATH_INFO: `/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:17.020874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] RBAC: Authorizing `identity:get_user(user_id=81dd7eeb5e2f48888fc0a98f63edea3c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:17.022365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:17.024014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 598/1182] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:11:16 2022] GET /identity/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:17.029807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:17.044961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:17.129701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab920-058d-41e3-9108-4ecb26986b11 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:17.162866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:17.163507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:17.163791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:17.164067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:17.228975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:17.230382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:17.243843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:17.326712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 585/1183] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:11:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:11:17.332629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:17.432198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390410-0d50-46c8-a8ec-3c7955d37b6e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:17.457902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:17.465809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:17.469323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:17.469323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:17.469323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] PATH_INFO: `/v3/projects/143cef91ae024ac9a7678e1098a5cdb8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:17.469323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] RBAC: Authorizing `identity:get_project(project_id=143cef91ae024ac9a7678e1098a5cdb8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:17.469951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7cbc-7408-407f-b683-c60d687410dd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:17.471805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 599/1184] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:11:17 2022] GET /identity/v3/projects/143cef91ae024ac9a7678e1098a5cdb8 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:17.481306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:17.590829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f86df-ac30-4c88-a12b-c94371f14306 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:17.604352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:17.625059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:17.625712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:17.626029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:17.626313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:17.627147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=143cef91ae024ac9a7678e1098a5cdb8, user.id=81dd7eeb5e2f48888fc0a98f63edea3c) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:17.627464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:17.628898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262d54-73b0-4230-ba7c-02d537197eae None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:17.634306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 586/1185] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:11:17 2022] GET /identity/v3/role_assignments?user.id=81dd7eeb5e2f48888fc0a98f63edea3c&scope.project.id=143cef91ae024ac9a7678e1098a5cdb8&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:19.661867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd624a-54d6-45ed-ab5f-ed5e57ab0229 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:19.662413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd624a-54d6-45ed-ab5f-ed5e57ab0229 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:19.662413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd624a-54d6-45ed-ab5f-ed5e57ab0229 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:19.663799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 600/1186] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:19.669205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:19.669832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:19.670029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:19.686936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:19.688133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:19.754554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 587/1187] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:19.778069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd624a-54d6-45ed-ab5f-ed5e57ab0229 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:19.883616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd624a-54d6-45ed-ab5f-ed5e57ab0229 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:19.919415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:19.920127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:19.920417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:19.920699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:19.931696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:11:19.939550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:11:19.940627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:11:19.941525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 601/1188] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:11:19.947661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:20.048811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35f04-8b11-40e2-8199-49d79ad71f05 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:20.056684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:20.084874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:20.085542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:20.085877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:20.086163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:20.086915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:20.087225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:20.088600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:20.096642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 588/1189] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:11:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:20.102463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:20.158198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:20.203862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ba73-a42c-45a6-b75f-a4e23468ebec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:20.236537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:20.237404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:20.237694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:20.238007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] PATH_INFO: `/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:20.239338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] RBAC: Authorizing `identity:get_user(user_id=81dd7eeb5e2f48888fc0a98f63edea3c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:20.241322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:20.243245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 602/1190] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:11:20 2022] GET /identity/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:20.248925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:20.346078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae569ec-0fbe-4ba1-a321-f858a1909a6a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:20.378948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:20.379797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:20.380078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:20.380350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] PATH_INFO: `/v3/projects/143cef91ae024ac9a7678e1098a5cdb8` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:20.381511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=143cef91ae024ac9a7678e1098a5cdb8)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:20.382954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec0f42-abcf-43ce-85ca-e54c5b943fe2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:20.384627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 589/1191] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:11:20 2022] GET /identity/v3/projects/143cef91ae024ac9a7678e1098a5cdb8 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:20.390563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:20.486020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc211a-f294-4d3c-bcbb-28ba6f388788 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:20.508341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:20.518586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:20.519463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:20.519742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:20.520016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] PATH_INFO: `/v3/projects/143cef91ae024ac9a7678e1098a5cdb8/users/81dd7eeb5e2f48888fc0a98f63edea3c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:20.522036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] RBAC: Authorizing `identity:create_grant(project_id=143cef91ae024ac9a7678e1098a5cdb8, user_id=81dd7eeb5e2f48888fc0a98f63edea3c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:20.523637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c0e08-e57d-4a04-8e0a-0d55c1f0301c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:20.676570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 603/1192] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:11:20 2022] PUT /identity/v3/projects/143cef91ae024ac9a7678e1098a5cdb8/users/81dd7eeb5e2f48888fc0a98f63edea3c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:11:22.732083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525e7b9-06dc-4c94-9fdf-91115d006cfb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:22.732625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525e7b9-06dc-4c94-9fdf-91115d006cfb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:22.732625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525e7b9-06dc-4c94-9fdf-91115d006cfb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:22.734049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 590/1193] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:11:22.739248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:22.739667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:22.739825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:22.757123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:11:22.758375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:11:22.824552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 604/1194] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:11:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:11:22.848105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525e7b9-06dc-4c94-9fdf-91115d006cfb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:22.885040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:22.950111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525e7b9-06dc-4c94-9fdf-91115d006cfb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:22.993366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:22.994179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:22.994467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:22.994744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.006508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:11:23.014039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:11:23.015133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:11:23.016333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 591/1195] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:11:23.021300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:23.121060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd5479-8558-4f84-a0e6-41bdeea2d38e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:23.137619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:23.159852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:23.160545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:23.160836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:23.161119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.161973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:23.162335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:23.163705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:23.173542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 605/1196] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:11:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:23.180203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:23.276325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:23.281926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196fb3-fed7-432e-808d-0c305add681c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:23.316483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:23.317345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:23.317628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:23.317943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] PATH_INFO: `/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.319147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] RBAC: Authorizing `identity:get_user(user_id=81dd7eeb5e2f48888fc0a98f63edea3c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:23.320568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:23.322222 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 592/1197] 10.222.0.93 () {56 vars in 1177 bytes} [Thu Jul 14 14:11:23 2022] GET /identity/v3/users/81dd7eeb5e2f48888fc0a98f63edea3c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:23.328115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:23.434938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98d646-07b7-412d-9443-13cea26ca831 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:23.469107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:23.469842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:23.470133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:23.470415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.516293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:23.539320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:23.540701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:23.642737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 606/1198] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:11:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:11:23.649184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:23.748588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25c5e5-406c-4f9e-8f65-2d06e00a4b72 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:23.782533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:23.783391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:23.783706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:23.783960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] PATH_INFO: `/v3/projects/143cef91ae024ac9a7678e1098a5cdb8` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.785148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] RBAC: Authorizing `identity:get_project(project_id=143cef91ae024ac9a7678e1098a5cdb8)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:23.786614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061bc2d-c639-4ad2-9abc-9e5a7c4b046f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:23.788877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 593/1199] 10.222.0.93 () {56 vars in 1186 bytes} [Thu Jul 14 14:11:23 2022] GET /identity/v3/projects/143cef91ae024ac9a7678e1098a5cdb8 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:23.795967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:11:23.872306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:23.898487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314caeec-0bb4-4b6d-900c-a4274aba27ec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:11:23.933118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:11:23.933803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:11:23.934085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:11:23.934367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:11:23.935459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=143cef91ae024ac9a7678e1098a5cdb8, user.id=81dd7eeb5e2f48888fc0a98f63edea3c) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:11:23.935776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:11:23.937255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b10ad-2bfc-471b-8519-882a1c131c02 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:11:23.943629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 607/1200] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:11:23 2022] GET /identity/v3/role_assignments?user.id=81dd7eeb5e2f48888fc0a98f63edea3c&scope.project.id=143cef91ae024ac9a7678e1098a5cdb8&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:11:33.778246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:11:33.940417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=30 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:39.924640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573da-ebe9-474a-b40c-1a387b801a19 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:39.925269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573da-ebe9-474a-b40c-1a387b801a19 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:39.925269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573da-ebe9-474a-b40c-1a387b801a19 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:39.926614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 594/1201] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:12:39.932064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:39.932712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:39.932712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:39.950757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:12:39.951760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:12:40.032245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 608/1202] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:12:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 14:12:40.064498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f573da-ebe9-474a-b40c-1a387b801a19 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:40.183758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f573da-ebe9-474a-b40c-1a387b801a19 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:40.228821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:40.229696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:40.230044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:40.230360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:40.241408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:40.242995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:40.262015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:40.263313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:40.264315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:40.264315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:40.264315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:40.264315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:40.264315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 595/1203] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:40.269909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:40.371300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28577538-22aa-400e-ae3f-20ea533ec121 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:40.402692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:40.408661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:40.409336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:40.409621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:40.409935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:40.410670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:40.410981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:40.412414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa566573-3d53-43f0-97ac-22c80952228f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:40.424914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 609/1204] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:40.430763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:40.490775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:40.539274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16305-a105-4790-8f68-fe195d950db7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:40.573530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:40.574235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:40.574519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:40.574789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:40.575498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:40.576956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae965c0-1d75-401d-bdca-f6907aaa1a73 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:40.679087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 596/1205] 10.222.0.93 () {60 vars in 1132 bytes} [Thu Jul 14 14:12:40 2022] POST /identity/v3/users => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:12:42.790132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15465-016d-43c6-aed4-20b9cf6f6089 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:42.790896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15465-016d-43c6-aed4-20b9cf6f6089 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:42.790896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15465-016d-43c6-aed4-20b9cf6f6089 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:42.792101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 610/1206] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:12:42.797502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:42.798000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:42.798100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:42.816819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:12:42.818068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:12:42.895579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 597/1207] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:12:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:12:42.932691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b15465-016d-43c6-aed4-20b9cf6f6089 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.040019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b15465-016d-43c6-aed4-20b9cf6f6089 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.076803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.077617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.077947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.078230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.092885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:12:43.100794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:43.101919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:43.102842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 611/1208] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:12:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:43.108304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.149968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:43.212177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0274-ffbd-4ec1-b8fb-7a1c2b612a2a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.246623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.247304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.247596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.247880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.248638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:43.248956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:43.251010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:43.259313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 598/1209] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:43.265243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.319339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:43.373352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e911-8e93-4532-8e52-311474979ac6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.408256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.409050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.409336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.409614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.420501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:43.421984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:43.441884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:43.443032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:43.443942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:43.443942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:43.443942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:43.443942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:43.443942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 612/1210] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:43.449048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.580594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:43.583287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fd8ef-c32d-4b67-a030-d4864e9180d0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.619838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.620754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.621243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.622023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.622771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:43.623077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:43.624518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:43.636432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 599/1211] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:43.642396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.757449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c181fd-1a38-4296-9203-025d4917d94b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.759064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:43.804044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.804850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.805196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.805478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.813409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:43.817078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:12:43.836655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:43.838637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:43.841892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 613/1212] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:43.846686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:43.954976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0639-aca1-4825-bc4f-649647e7e5dd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:43.993563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:43.994260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:43.994565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:43.994865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:43.995717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:43.996029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:43.997469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:44.015430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 600/1213] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:12:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:44.021828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:44.162721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f68f-3298-4be8-a2df-f6c9830a695a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:44.210699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:44.211370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:44.211660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:44.211941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:44.247857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:44.288937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:44.290434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:44.394362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 614/1214] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:12:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:12:44.400620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:44.420219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:44.509507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407c3a-113e-4ee7-9a6d-1faa5b2b5811 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:44.544930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:44.545798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:44.546074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:44.546351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:44.556788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:44.558405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:44.576676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:44.577889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:44.579229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:44.579229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:44.579229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:44.579229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:44.579229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 601/1215] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:44.583862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:44.646823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:44.687636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd794a-8804-46e1-99e2-3d1b3f0140a9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:44.724436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:44.725112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:44.725396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:44.725674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:44.726438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:44.726744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:44.728108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:44.739658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 615/1216] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:44.745364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:44.822806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:44.854302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432c76-5970-47f3-960f-d0de8285b89d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:44.887670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:44.891448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:44.891448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:44.891448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:44.896354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:44.898039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:12:44.912692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:44.913756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:44.914819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 602/1217] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:44.919912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:45.038950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef15b-628b-413c-9b78-ce3ba5e9b8d3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:45.062516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:45.074595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:45.075429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:45.075719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:45.076000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:45.076806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:45.077118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:45.079185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13abca-668a-4dab-b5c6-06513ae78df4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:45.092615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 616/1218] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:45.099341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:45.202819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea655f0-5fb8-41eb-8baf-dee166b296c6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:45.233629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:45.240090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:45.240752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:45.241040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:45.241322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:45.242208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=768f4ab96bcf430f8e26b4d08d49581a) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:45.242526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:45.243977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a05a-6ab5-4bf1-9ce0-d7117c81e319 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:45.249690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 603/1219] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:12:45 2022] GET /identity/v3/role_assignments?user.id=768f4ab96bcf430f8e26b4d08d49581a&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:47.303662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed092-7263-474d-811d-345ed7106e30 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:47.304224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed092-7263-474d-811d-345ed7106e30 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:47.304224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed092-7263-474d-811d-345ed7106e30 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:47.305591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 617/1220] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:12:47.311448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:47.312039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:47.312176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:47.329393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:12:47.332707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:12:47.400412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 604/1221] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:12:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:12:47.425383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed092-7263-474d-811d-345ed7106e30 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:47.529610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed092-7263-474d-811d-345ed7106e30 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:47.567489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:47.568235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:47.568519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:47.568794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:47.578874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:12:47.586679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:47.587784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:47.588839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 618/1222] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:47.594568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:47.694234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1c461-3a27-4608-bc61-5a2863282f14 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:47.718269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:47.729424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:47.730141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:47.730433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:47.733869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:47.733869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:47.733869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:47.733869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:47.741681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 605/1223] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:47.747442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:47.796847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:47.859363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0bc0d-5850-4a19-ae90-d2d9faa158db None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:47.894201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:47.894992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:47.895287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:47.895543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:47.907895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:47.909404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:47.926821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:47.927924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:47.928908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:47.928908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:47.928908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:47.928908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:47.928908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 619/1224] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 14:12:47.934266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.040501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c761222-1bb5-4c5a-aa1b-b24c3400616e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.076703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.077595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.078721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.079114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.079996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:48.080428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.082196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.099314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 606/1225] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:48.105572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.213283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:48.230115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c586e26-f8da-47a5-9528-0192aba34b4a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.264960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.265732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.266049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.266331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.274430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.275843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:12:48.291345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:48.292554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:48.293516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 620/1226] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:48.299355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.349277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:48.405946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0a3c3-b042-46f3-9080-7e445c24d4f9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.440855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.441527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.441840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.442121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.442971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:48.443279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.444881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.462995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 607/1227] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:48.470859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.571494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7882c74-0fe2-48d9-a20e-6897e8cbbf79 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.606724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.607592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.607875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.608154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.619113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.620548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:48.637431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:48.638582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:48.639730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:48.639730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:48.639730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:48.639730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:48.639730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 621/1228] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:48.644967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.753837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fcd4-0a69-4673-a27f-51f05991cc44 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.756313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:48.788139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.788826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.789110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.789390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.790311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:48.790633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.791996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.803636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 608/1229] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:48.809601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:48.820689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:48.919641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ced73f-520f-4597-a7a1-e88e5a97c32b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:48.956911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:48.957818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:48.958108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:48.958390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:48.966499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:48.968372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:12:48.983363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:48.984580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:48.985698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 622/1230] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:48.997837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:49.099819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2a861-6f12-41a4-8a9f-ad3142a6a6cd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:49.136384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:49.137103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:49.137385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:49.137666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:49.138508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:49.138823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:49.140267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18aedb-aa90-4713-95a6-6a7b537c2b9c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:49.152814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 609/1231] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:49.159481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:49.229935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:49.265536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683821-0e91-4304-b6db-8d802c275229 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:49.301499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:49.302587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:49.302871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:49.303169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/768f4ab96bcf430f8e26b4d08d49581a/roles/c01dbafd47b1456f991c9fc8c7e1dd65` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:49.321688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=768f4ab96bcf430f8e26b4d08d49581a, role_id=c01dbafd47b1456f991c9fc8c7e1dd65)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:49.323058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d2901-2167-4f79-ad1e-8836b0d1a759 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:49.401601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 623/1232] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:12:49 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/768f4ab96bcf430f8e26b4d08d49581a/roles/c01dbafd47b1456f991c9fc8c7e1dd65 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:12:51.425096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8f6a-9092-4823-b786-2b7dd1099d9d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:51.425638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8f6a-9092-4823-b786-2b7dd1099d9d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:51.425638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8f6a-9092-4823-b786-2b7dd1099d9d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:51.426942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 610/1233] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:12:51.433662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:51.434263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:51.434586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:51.452799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:12:51.454064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:12:51.527615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 624/1234] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:12:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:12:51.552099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff8f6a-9092-4823-b786-2b7dd1099d9d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:51.579359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:51.659602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff8f6a-9092-4823-b786-2b7dd1099d9d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:51.694918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:51.695630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:51.695915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:51.696193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:51.706205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:12:51.713987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:51.715040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:51.715893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 611/1235] 10.222.0.93 () {56 vars in 1102 bytes} [Thu Jul 14 14:12:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:51.721405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:51.769638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:51.837511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871331ad-fded-4e20-ad39-00799ab170f0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:51.874747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:51.875429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:51.875712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:51.875990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:51.876747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:51.877057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:51.878505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:51.887562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 625/1236] 10.222.0.93 () {56 vars in 1103 bytes} [Thu Jul 14 14:12:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:51.893538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:51.988481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:52.004228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89280fd-23e5-40ea-b3dc-0c0d65b79333 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:52.038120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:52.038911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:52.039195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:52.039477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:52.050899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:52.052360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:52.071027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:52.072133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:52.073390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:52.073390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:52.073390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:52.073390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:52.073390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 612/1237] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:52.078568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:52.187796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b395-6555-4db0-9ba8-24b5a711e32b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:52.197888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:52.226289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:52.226965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:52.227252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:52.227535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:52.228277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:52.228592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:52.230000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:52.243938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 626/1238] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:52.250221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:52.362824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22651301-0a9d-404b-bb0d-ecb9d4969054 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:52.398908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:52.399993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:52.400264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:52.400542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:52.403365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:52.408771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:52.410259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:12:52.434273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:52.436231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:52.437553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 613/1239] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:52.442170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:52.546903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e3b76-83b1-4af4-b086-6e1494438480 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:52.585227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:52.585955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:52.586244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:52.586529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:52.587404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:52.587718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:52.589949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:52.613158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 627/1240] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:12:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:52.619345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:52.730966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640bc3-1113-4ab1-bf7b-7d732559c06d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:52.768652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:52.769305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:52.769594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:52.769911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:52.829857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:52.836303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:52.837777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:52.941972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 614/1241] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:12:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:12:52.944898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:52.976057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:53.057105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33514770-5a11-4a8c-94ef-00c8b3869fd3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:53.102800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:53.103525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:53.103858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:53.104138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:53.114820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:53.116378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:53.134935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:53.136047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:53.137368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:53.137368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:53.137368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:53.137368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:53.137368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 628/1242] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:53.142542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:53.235290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:53.240723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45685f19-9909-4af2-b385-398e11213a38 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:53.274275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:53.274973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:53.275257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:53.275539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:53.276265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:53.276576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:53.277975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:53.289146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 615/1243] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:53.294955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:53.382348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:53.404056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d22f4-b065-4fd0-986d-6d34e9ea79c3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:53.441108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:53.442050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:53.442337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:53.442893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:53.450474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:53.451925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:12:53.467130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:53.468296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:53.469355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 629/1244] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:53.474892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:53.587640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf432691-74d4-48ca-8e93-1795f218939d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:53.623177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:53.623841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:53.624130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:53.624411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:53.625793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:53.626121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:53.627561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b63c1-b318-49d4-a8b0-a723ee421680 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:53.640477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 616/1245] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:12:53.646434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:53.751351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79de8d-4317-4bc8-974b-514ef0455781 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:53.785486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:53.786227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:53.786513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:53.786791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:53.787619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] RBAC: Adding query filter params (role.id=c01dbafd47b1456f991c9fc8c7e1dd65, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=768f4ab96bcf430f8e26b4d08d49581a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:53.787936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:53.789364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba993aad-11b7-4f67-81fb-ebaccee688f1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:53.795760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 630/1246] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:12:53 2022] GET /identity/v3/role_assignments?user.id=768f4ab96bcf430f8e26b4d08d49581a&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=c01dbafd47b1456f991c9fc8c7e1dd65 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:55.942403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff90eb-60f9-469c-9636-92ae5ba6ee83 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:55.943014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff90eb-60f9-469c-9636-92ae5ba6ee83 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:55.943014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff90eb-60f9-469c-9636-92ae5ba6ee83 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:55.944307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 617/1247] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:12:55.949619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:55.950089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:55.950256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:55.955666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:55.967844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:12:55.969075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:12:56.041004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 631/1248] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:12:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:12:56.066511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff90eb-60f9-469c-9636-92ae5ba6ee83 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:56.084999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:56.176466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff90eb-60f9-469c-9636-92ae5ba6ee83 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:56.212469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:56.212469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:56.212469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:56.212469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:56.227663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:12:56.236027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:56.238244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:56.239302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 618/1249] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:56.243534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:56.349979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0377b-2826-4ea2-9194-1a5d227f8d5e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:56.380151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:56.385125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:56.385814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:56.386100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:56.386374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:56.387120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:56.387426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:56.388798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:56.397484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 632/1250] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:56.403847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:56.486748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:56.509943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d473-1b6e-4866-b232-c7c511db02c3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:56.544494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:56.545293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:56.545589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:56.545896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:56.556831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:56.558366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:56.576861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:56.577922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:56.579018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:56.579018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:56.579018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:56.579018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:56.579018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 619/1251] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:56.583974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:56.712221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc57f4-95f9-4eec-ae62-ea9ae2039a4f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:56.749569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:56.750219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:56.750507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:56.750791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:56.751523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:56.751836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:56.753208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:56.766226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 633/1252] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:56.772230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:56.875351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf9425-c324-4f07-b314-503499a22f5c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:56.902484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:56.910715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:56.911511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:56.911795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:56.912069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:56.919643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:56.921135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:12:56.936859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:56.938144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:12:56.939408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 620/1253] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:56.944491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:57.017862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:57.068962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d3f61-758f-4a0b-8474-30fccd163883 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:57.111250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:57.112505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:57.112794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:57.113296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:57.114194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:57.114506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:57.115945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:57.134929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 634/1254] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:12:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:57.141443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:57.258254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57dcb-4d51-41ce-a641-e64d1e24cde6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:57.301961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:57.301961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:57.301961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:57.301961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:57.364365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:57.378057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:57.379529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:57.486961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 621/1255] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:12:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 14:12:57.493387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:57.600964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99e21-860d-4dc9-88fd-22a78cb9c758 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:57.614910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:57.643381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:57.644180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:57.644488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:57.644773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:57.654689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:57.656196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:12:57.676117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:57.677068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:57.678139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:12:57.678139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:12:57.678139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:12:57.678139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:57.678139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 635/1256] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:57.683839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:57.774538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:57.787026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827eaed-09ad-4aa2-abe0-e02eb5dd0fe2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:57.822845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:57.823509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:57.823796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:57.824077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:57.824808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:57.825114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:57.826506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:57.837747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 622/1257] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:57.845260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:57.950929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d33b0-e9bd-4615-97c7-fc083370d004 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:57.986711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:57.987508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:57.987793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:57.988068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:57.995052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:57.996601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:12:58.010602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:12:58.012240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:12:58.014069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 636/1258] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:12:58.017865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:58.125994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2af92-bd60-4e09-8858-0e7c87790d89 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:58.160255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:58.160949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:58.161204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:58.161485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:58.162372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:58.162683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:58.164128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff2c5-fdc2-4443-b139-c255c544fca5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:58.176824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 623/1259] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:12:58.183107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:12:58.211022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:12:58.295709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d30e4-deb1-4c78-ba85-aff603702279 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:12:58.330643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:12:58.331293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:12:58.331575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:12:58.331845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:12:58.332686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=768f4ab96bcf430f8e26b4d08d49581a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:12:58.333000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:12:58.334496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad8e34-7aab-4d0a-b893-762e3aeb391a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:12:58.340599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 637/1260] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:12:58 2022] GET /identity/v3/role_assignments?user.id=768f4ab96bcf430f8e26b4d08d49581a&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:00.383217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d35-7e82-4473-8a62-ff41516e482c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:00.383852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d35-7e82-4473-8a62-ff41516e482c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:00.383852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d35-7e82-4473-8a62-ff41516e482c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:00.385377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 624/1261] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:13:00.391692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:00.392236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:00.392236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:00.412932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:00.414327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:00.484298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 638/1262] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:00.507258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f23d35-7e82-4473-8a62-ff41516e482c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:00.519131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:00.618264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f23d35-7e82-4473-8a62-ff41516e482c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:00.653011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:00.653709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:00.654030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:00.654310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:00.665226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:13:00.673040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:00.677258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:00.679143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 625/1263] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:13:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:00.680340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:00.699471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:00.783105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b12d0a-8711-4a43-bf73-3515f9b1c50e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:00.821889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:00.822571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:00.822858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:00.823139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:00.823904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:00.824223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:00.825606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:00.834837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 639/1264] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:13:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:00.840560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:00.909119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:00.945877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e453c-c60a-46d1-912c-17236595f583 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:00.985249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:00.986076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:00.986367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:00.992122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.000082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.000730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:13:01.024539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:01.025747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:01.026989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:13:01.026989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:13:01.026989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:01.026989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:01.026989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 626/1265] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:01.036742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.123852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:01.143926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb877-3f1f-4229-b09d-1b89d3a305a1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:01.181133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:01.181715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:01.182028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:01.182304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.183042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:01.183350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.184707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.197986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 640/1266] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:01.203809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.313838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489f836-a6df-4bf8-9e0e-b52b4997c528 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:01.333914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:01.347928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:01.348735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:01.349027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:01.349310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.357033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.359248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:13:01.375008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:01.376167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:01.377436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 627/1267] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:01.382379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.487174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554d75-b6e2-4578-8260-d9659b5b82d1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:01.526190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:01.526861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:01.527252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:01.527424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.528269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:01.528581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.530045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.539869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:01.550065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 641/1268] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:01.555860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.667202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7242-bebb-494c-8727-8aed7374ed98 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:01.701985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:01.702776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:01.703061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:01.703345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.714537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.715984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:13:01.734004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:01.735158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:01.736119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:13:01.736119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:13:01.736119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:01.736119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:01.736119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 628/1269] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:01.741133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.846008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e44a9-b97e-4ba9-8d97-cb467ba58bda None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:01.882407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:01.883068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:01.883354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:01.883663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:01.884503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:01.884814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:01.886473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:01.898842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 642/1270] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:01.904517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:01.937461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:02.005878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77202bf8-3fe5-4408-bd2f-5d066981d7b0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:02.041662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:02.042480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:02.042762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:02.043032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:02.049890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:02.051378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:13:02.065311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:02.066413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:02.067377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 629/1271] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:02.072650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:02.138190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:02.184606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eed05-e221-44e2-aa24-f391593ff04f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:02.222395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:02.223090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:02.223377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:02.223654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:02.224660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:02.224971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:02.226448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba722209-208f-45ba-9d20-129848b44430 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:02.238933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 643/1272] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:13:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:13:02.245186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:02.338155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:02.346556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b0888-9696-48d0-86d5-0b16575aa10a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:02.380724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:02.381623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:02.381936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:02.382213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] PATH_INFO: `/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/768f4ab96bcf430f8e26b4d08d49581a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:02.384234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe18e4150846cb89e1e269df5dcd77, user_id=768f4ab96bcf430f8e26b4d08d49581a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:02.385674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500b7bc-6d08-444a-8198-aefd7ebfa5f4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:02.474995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 630/1273] 10.222.0.93 () {58 vars in 1439 bytes} [Thu Jul 14 14:13:02 2022] PUT /identity/v3/projects/0bbe18e4150846cb89e1e269df5dcd77/users/768f4ab96bcf430f8e26b4d08d49581a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:13:04.541404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1c7e0-ec71-4791-936c-4f19b56d42cf None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:04.541978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1c7e0-ec71-4791-936c-4f19b56d42cf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:04.542064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1c7e0-ec71-4791-936c-4f19b56d42cf None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:04.543423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 644/1274] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:13:04.550007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:04.550543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:04.550543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:04.567808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:04.569004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:04.639357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 631/1275] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:04.663468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1c7e0-ec71-4791-936c-4f19b56d42cf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:04.784785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1c7e0-ec71-4791-936c-4f19b56d42cf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:04.802504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:04.819378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:04.820186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:04.820471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:04.820749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:04.832618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 14:13:04.840477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:04.841717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:04.843490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 645/1276] 10.222.0.93 () {56 vars in 1096 bytes} [Thu Jul 14 14:13:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:04.848347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:04.936973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:04.955320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb6e7a-6be6-4566-aaeb-a6d7b1b4be68 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:04.993798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:04.994473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:04.994758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:04.995036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:04.995862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:04.996173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:04.997553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:05.007250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 632/1277] 10.222.0.93 () {56 vars in 1099 bytes} [Thu Jul 14 14:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:05.013365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:05.133489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282504e-1677-43f5-983e-23d98541df3c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:05.169495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:05.170362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:05.170675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:05.170959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:05.185800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:05.187316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:13:05.206429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:05.207588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:05.208985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:13:05.208985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:13:05.208985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:05.208985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:05.208985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 646/1278] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:05.213710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:05.320083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367716e9-0d46-4b9b-a868-b1348943237b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:05.354990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:05.355652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:05.355942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:05.356225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:05.356955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:05.357265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:05.358670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:05.370594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 633/1279] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:05.376890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:05.411680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:05.483916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214470a-086b-4b05-8d8d-503468cbabf1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:05.520960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:05.521850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:05.522146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:05.522502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:05.530533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:05.531984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 14:13:05.546483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:05.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:05.549305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 647/1280] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:05.554391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:05.557890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:05.656084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe076a3-d7b1-424a-b97c-84faf97bf0cf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:05.690829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:05.691497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:05.691783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:05.692064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:05.692900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:05.693211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:05.694820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:05.712729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 634/1281] 10.222.0.93 () {56 vars in 1143 bytes} [Thu Jul 14 14:13:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:05.718617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:05.815945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:05.826822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c8cd7-2c97-4770-acae-c50d42466251 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:05.861859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:05.862529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:05.862818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:05.863100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:05.932116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:05.933472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.035345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:06.050761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 648/1282] 10.222.0.93 () {58 vars in 1282 bytes} [Thu Jul 14 14:13:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 14:13:06.057124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:06.121262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:06.159551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf77a-9e92-47cd-94c9-c9b912b183c2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:06.193840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:06.194643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:06.194961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:06.195294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:06.206140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:06.207657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 14:13:06.225257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:06.226442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:06.227459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 14:13:06.227459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 14:13:06.227459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 14:13:06.227459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:06.227459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 635/1283] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:06.232449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:06.334653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66312-8af3-45d2-aa84-c152da5fb043 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:06.371725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:06.372396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:06.372685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:06.372965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:06.373721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:06.374062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:06.375439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.391245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 649/1284] 10.222.0.93 () {56 vars in 1109 bytes} [Thu Jul 14 14:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:06.397413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:06.495916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd53196-9f49-4ed5-9ddb-9c4695054d94 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:06.509076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:06.528656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:06.529445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:06.529798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:06.530652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:06.537615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:06.539143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 14:13:06.554218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:06.556377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Jul 14 14:13:06.558084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 636/1285] 10.222.0.93 () {56 vars in 1111 bytes} [Thu Jul 14 14:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:06.561676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:06.612125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:06.660872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371028a-82a5-4052-bd32-22a0bfaba492 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:06.695137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:06.695794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:06.696086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:06.696369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:06.697162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:06.697476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:06.699160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcedc9-d3b5-4f8f-988a-a433c8352843 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.711897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 650/1286] 10.222.0.93 () {56 vars in 1148 bytes} [Thu Jul 14 14:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:06.719063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:06.818592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492fc8-9d4f-490a-8590-48a674191b00 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:06.853792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:06.854478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:06.854794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:06.855078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:06.855981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] RBAC: Adding query filter params (role.id=df6ee0fb73ff4f50bbefafd179e47864, scope.project.id=0bbe18e4150846cb89e1e269df5dcd77, user.id=768f4ab96bcf430f8e26b4d08d49581a) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:06.856336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:06.857814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f15c92-5eea-4cfe-b7c5-a5847a7e6ee8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:06.863976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 637/1287] 10.222.0.93 () {56 vars in 1374 bytes} [Thu Jul 14 14:13:06 2022] GET /identity/v3/role_assignments?user.id=768f4ab96bcf430f8e26b4d08d49581a&scope.project.id=0bbe18e4150846cb89e1e269df5dcd77&role.id=df6ee0fb73ff4f50bbefafd179e47864 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:08.996493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270caf-f959-46dd-9562-0af0ab782978 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:08.997150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270caf-f959-46dd-9562-0af0ab782978 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:08.997150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270caf-f959-46dd-9562-0af0ab782978 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:08.999074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 651/1288] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:13:09.004428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:09.004895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:09.004999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:09.022325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:09.023516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:09.058857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:09.101794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 638/1289] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:09.125659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270caf-f959-46dd-9562-0af0ab782978 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:09.232044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270caf-f959-46dd-9562-0af0ab782978 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:09.244104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:09.269225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:09.269951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:09.270228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:09.270510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:09.271158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:09.272522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:09.284437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:09.287696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:09.288800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 652/1290] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:13:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:09.292157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:09.402211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5023d-0228-4801-9da6-e99f34014d0c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:09.441434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:09.442130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:09.442413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:09.442697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:09.443403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:09.443710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:09.445092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca3d38-e685-4fd7-9426-0241645f8c93 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:09.450501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 639/1291] 10.222.0.93 () {56 vars in 1116 bytes} [Thu Jul 14 14:13:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:09.456534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:09.578187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b96c-17a3-4338-9136-7b94aa9ec680 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:09.618385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:09.619140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:09.619437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:09.619720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:09.620465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:09.620777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:09.622255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8d33-e0bc-4e10-9836-fab1eb42de49 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:09.627711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 653/1292] 10.222.0.93 () {56 vars in 1116 bytes} [Thu Jul 14 14:13:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:11.722118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb6e9-0e5f-440e-94de-5ffe957b3b21 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:11.722691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb6e9-0e5f-440e-94de-5ffe957b3b21 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:11.722691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb6e9-0e5f-440e-94de-5ffe957b3b21 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:11.724118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 640/1293] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:13:11.729844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8eda0-5123-4d38-9d8e-f193ea0f7acd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:11.730343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8eda0-5123-4d38-9d8e-f193ea0f7acd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:11.730506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8eda0-5123-4d38-9d8e-f193ea0f7acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:11.748911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8bb8eda0-5123-4d38-9d8e-f193ea0f7acd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:11.754422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8bb8eda0-5123-4d38-9d8e-f193ea0f7acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:11.780330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:11.819869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 654/1294] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:11.843072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cb6e9-0e5f-440e-94de-5ffe957b3b21 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:11.878245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:11.955236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cb6e9-0e5f-440e-94de-5ffe957b3b21 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:11.990769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:11.991444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:11.991727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:11.992011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:11.992737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:11.994151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abbb73-e215-49b3-8075-b790292ab298 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:12.109330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 641/1295] 10.222.0.93 () {60 vars in 1141 bytes} [Thu Jul 14 14:13:11 2022] POST /identity/v3/services => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:13:14.196257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97e42-a879-42dd-88e1-6b29583adfca None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.196825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97e42-a879-42dd-88e1-6b29583adfca None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.196825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97e42-a879-42dd-88e1-6b29583adfca None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.198229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 655/1296] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:13:14.203690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.204155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.204289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.222103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:14.223315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:14.303639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 642/1297] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:14.329479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97e42-a879-42dd-88e1-6b29583adfca None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:14.430823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97e42-a879-42dd-88e1-6b29583adfca None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:14.468010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:14.468811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.469090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.469363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.470038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:14.471409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:14.484010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:14.487482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:14.488401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 656/1298] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:13:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:14.491271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:14.510853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:14.596139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cbb00-47a7-4f75-be3d-1c5797fcc2b6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:14.636042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:14.636716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.637006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.637294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.638038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:14.638349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:14.640464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:14.646042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 643/1299] 10.222.0.93 () {56 vars in 1116 bytes} [Thu Jul 14 14:13:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:14.652211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:14.685990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:14.754713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679930ec-84b7-40a8-ae3e-4d5df162e4c0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:14.791112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:14.791779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.792068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.792398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.793159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] RBAC: Adding query filter params (interface=public, service_id=e58fb21176b449708870f2c5c40be1df, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:14.793471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:14.794876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586976c2-d549-4926-8d11-96a4a3b9fa10 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:14.800076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 657/1300] 10.222.0.93 () {56 vars in 1251 bytes} [Thu Jul 14 14:13:14 2022] GET /identity/v3/endpoints?service_id=e58fb21176b449708870f2c5c40be1df&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:14.806073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:14.899682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:14.918027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5c49-9ec1-43ca-a5f8-7afbee26ec8c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:14.957965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:14.958671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:14.958960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:14.959291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:14.960041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:14.961450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf6970-9cee-407d-99db-564c00c8661c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:14.969247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 644/1301] 10.222.0.93 () {56 vars in 1087 bytes} [Thu Jul 14 14:13:14 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:13:17.032932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a29f5-676b-4866-b0eb-83f619ae6762 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:17.033516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a29f5-676b-4866-b0eb-83f619ae6762 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:17.033516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a29f5-676b-4866-b0eb-83f619ae6762 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:17.034956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 658/1302] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:13:17.040556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:17.041026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:17.041190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:17.058445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:13:17.059912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:13:17.128019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 645/1303] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:13:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 14:13:17.152309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a29f5-676b-4866-b0eb-83f619ae6762 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:17.251277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:17.257962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a29f5-676b-4866-b0eb-83f619ae6762 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:17.293254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:17.294001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:17.294290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:17.294571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:17.295278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:17.296656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 14:13:17.308718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 14:13:17.310286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 14 14:13:17.311803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 659/1304] 10.222.0.93 () {56 vars in 1117 bytes} [Thu Jul 14 14:13:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 14:13:17.316974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:17.403970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:17.427951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75ef15-eb14-4b12-bc26-9dcc009ddcd6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:17.464591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:17.465270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:17.465559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:17.465908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:17.466685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:13:17.467015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:17.468400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41af569-0986-47d4-b03a-309d571802fd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:17.473912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 646/1305] 10.222.0.93 () {56 vars in 1116 bytes} [Thu Jul 14 14:13:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:13:17.480091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:13:17.582553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a691e-8131-43ac-b5a8-05a2953d5740 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:13:17.616370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:13:17.617026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:13:17.617420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:13:17.617711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:13:17.618490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:13:17.619916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad43c6-af2d-45e0-931f-b86203250d36 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:13:17.726897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 660/1306] 10.222.0.93 () {60 vars in 1144 bytes} [Thu Jul 14 14:13:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:13:27.470842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=30 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:13:27.721091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:05.722158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb9ec-4c31-4658-ae25-0ca5ddbf5bfe None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:05.722702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb9ec-4c31-4658-ae25-0ca5ddbf5bfe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:05.722702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb9ec-4c31-4658-ae25-0ca5ddbf5bfe None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:05.729291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204f74a-b032-4580-9dea-cdb78e7fcf82 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:05.729793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204f74a-b032-4580-9dea-cdb78e7fcf82 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:05.729882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204f74a-b032-4580-9dea-cdb78e7fcf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:05.733545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 647/1307] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:05.815479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c204f74a-b032-4580-9dea-cdb78e7fcf82 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:05.816695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c204f74a-b032-4580-9dea-cdb78e7fcf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:05.962325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 661/1308] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:22:06.000711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:06.000944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:06.001232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:06.002641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 648/1309] 10.222.0.93 () {54 vars in 964 bytes} [Thu Jul 14 14:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:06.009268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a55-f515-4538-9e7f-90aa3cee0e15 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:06.009582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a55-f515-4538-9e7f-90aa3cee0e15 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:06.009792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a55-f515-4538-9e7f-90aa3cee0e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:06.133856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c5a76a55-f515-4538-9e7f-90aa3cee0e15 None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:06.135316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5a76a55-f515-4538-9e7f-90aa3cee0e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:06.158297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:06.222593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 662/1310] 10.222.0.93 () {58 vars in 1061 bytes} [Thu Jul 14 14:22:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:22:06.232114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:06.233993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:06.346109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29c76f-f8ab-4e75-bc4b-d8fecfa7aa55 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:06.424705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:06.425360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:06.425646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:06.425955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:06.427611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:06.485241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:06.542016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:06.543461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a8226-722d-4aed-829c-b4461a2d0f96 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:06.563584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 649/1311] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:08.922730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead468-4f30-4618-9000-7bde2aca546a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:08.923286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead468-4f30-4618-9000-7bde2aca546a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:08.923286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead468-4f30-4618-9000-7bde2aca546a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:08.924925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 663/1312] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:08.930332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebda44-32a3-46c5-8238-57925b36b757 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:08.930794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebda44-32a3-46c5-8238-57925b36b757 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:08.930878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebda44-32a3-46c5-8238-57925b36b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:09.006321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-aaebda44-32a3-46c5-8238-57925b36b757 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:09.008085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aaebda44-32a3-46c5-8238-57925b36b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:09.131262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 650/1313] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:09.185944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:09.186385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:09.186681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:09.189115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 664/1314] 10.222.0.93 () {54 vars in 964 bytes} [Thu Jul 14 14:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:09.195608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc246f-de58-45b1-a9b4-a255ffed273f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:09.195918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc246f-de58-45b1-a9b4-a255ffed273f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:09.196199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc246f-de58-45b1-a9b4-a255ffed273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:09.207200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:09.292097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-69fc246f-de58-45b1-a9b4-a255ffed273f None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:09.293336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-69fc246f-de58-45b1-a9b4-a255ffed273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:09.371483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 651/1315] 10.222.0.93 () {58 vars in 1061 bytes} [Thu Jul 14 14:22:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:09.380790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:09.383577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:09.513745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d181-bcf9-440e-af72-934b25aa953f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:09.589750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:09.592150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:09.592809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:09.593094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:09.593371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:09.595069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:09.704409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:09.705791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb2df-d564-41ae-9c8b-18f3523a7f4c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:09.724239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 665/1316] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:12.523774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:12.524313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:12.524313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:12.525682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 652/1317] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:12.530754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda40a8-ee73-4cc4-90a3-1c7d27ee8962 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:12.531146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda40a8-ee73-4cc4-90a3-1c7d27ee8962 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:12.531323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda40a8-ee73-4cc4-90a3-1c7d27ee8962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:12.595948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0cda40a8-ee73-4cc4-90a3-1c7d27ee8962 None None] MFA Rules not processed for user `64e0572a2aa94e349233dd92e98da7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:12.597206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0cda40a8-ee73-4cc4-90a3-1c7d27ee8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:12.651482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:12.695102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 666/1318] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:12.734552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:12.737118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:12.841761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45a41-681a-4e0e-be4e-a91dc00f10fe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:12.844473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:12.915968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:12.916620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:12.916910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:12.917193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:12.918905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:13.026846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:13.028285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2db2-72ec-4d2f-ba7f-c18e538476e8 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:13.046782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 653/1319] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:15.906542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:15.907058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:15.907058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:15.908523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 667/1320] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:15.914389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e264-6c04-40a4-b5ac-9a29fcfbd828 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:15.914854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e264-6c04-40a4-b5ac-9a29fcfbd828 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:15.914947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e264-6c04-40a4-b5ac-9a29fcfbd828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:15.953698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:15.978649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5774e264-6c04-40a4-b5ac-9a29fcfbd828 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:15.979976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5774e264-6c04-40a4-b5ac-9a29fcfbd828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:16.077037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 654/1321] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:16.115790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:16.117336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:16.224530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4795e9f-f2cd-4956-9bd9-b95bfbd19766 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:16.296888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:16.297539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:16.297860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:16.298144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:16.299803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:16.307725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:16.408437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:16.409908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d66bc2-10f8-43a4-a7d2-0db973599288 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:16.428088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 668/1322] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:19.422359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:19.422870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:19.422870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:19.424233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 655/1323] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:19.429598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393154-dcd3-4c2e-a133-f9a1ba973c36 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:19.430051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393154-dcd3-4c2e-a133-f9a1ba973c36 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:19.430206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393154-dcd3-4c2e-a133-f9a1ba973c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:19.499095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-68393154-dcd3-4c2e-a133-f9a1ba973c36 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:19.500702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-68393154-dcd3-4c2e-a133-f9a1ba973c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:19.561464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:19.602683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 669/1324] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:19.639200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:19.640818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:19.755028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618925f-f2a3-4576-ba7b-8f1da4bf1b9c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:19.768718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:19.835732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:19.836394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:19.836678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:19.836956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:19.838703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:19.952234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:19.953681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c770-672d-4ec2-ab15-58ba43db3d3e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:19.971963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 656/1325] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:22.097375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:22.098002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:22.098002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:22.099364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 670/1326] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:22.105113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b56da-fc3e-44a5-8d96-37155f22d28c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:22.105571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b56da-fc3e-44a5-8d96-37155f22d28c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:22.105691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b56da-fc3e-44a5-8d96-37155f22d28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:22.153902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:22.173071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-087b56da-fc3e-44a5-8d96-37155f22d28c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:22.174354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-087b56da-fc3e-44a5-8d96-37155f22d28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:22.279590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 657/1327] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:22.316864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:22.318500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:22.430554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b761d6-c9d5-4654-b6cb-b1eacf611487 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:22.506147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:22.507351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:22.507643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:22.507919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:22.509571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:22.528459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:22.627566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:22.629079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69b04c-b8af-4d0b-9985-a610649f570e service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:22.651582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 671/1328] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:24.818490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:24.819577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:24.819577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:24.820969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 658/1329] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:24.826188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32304d-5bb4-4087-8839-a75894458059 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:24.826611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32304d-5bb4-4087-8839-a75894458059 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:24.826827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32304d-5bb4-4087-8839-a75894458059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:24.892161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ee32304d-5bb4-4087-8839-a75894458059 None None] MFA Rules not processed for user `64e0572a2aa94e349233dd92e98da7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:24.893373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ee32304d-5bb4-4087-8839-a75894458059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:24.968574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:25.004688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 672/1330] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:25.043182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:25.044833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:25.155374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce239a-edf5-4cfb-a7cc-a7b6778101fd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:25.176207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:25.231149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:25.231720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:25.232010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:25.232290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:25.233991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:25.346582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:25.348033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e73748-9095-492d-8b88-c49aced76d33 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:25.367149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 659/1331] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:28.070600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:28.071038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:28.071167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:28.072555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 673/1332] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:22:28.077702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb2b-9e06-484a-865f-4f74790f5f8c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:28.078203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb2b-9e06-484a-865f-4f74790f5f8c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:28.078308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb2b-9e06-484a-865f-4f74790f5f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:28.140518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:28.143429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0851fb2b-9e06-484a-865f-4f74790f5f8c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:28.144699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0851fb2b-9e06-484a-865f-4f74790f5f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:28.241897 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 660/1333] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:28.278953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:28.280525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:28.385916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7753fcb-c930-4824-b5f9-4dcf72ea0655 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:28.458222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:28.458987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:28.459350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:28.460126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:28.461803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:28.488561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:28.570649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:28.572082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f49b7b-9999-4c49-a8f2-ce31022e6d14 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:28.590393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 674/1334] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:30.670250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:30.670799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:30.670799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:30.672179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 661/1335] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:30.677558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4195-f659-44c5-a352-4a0190c13b29 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:30.678661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4195-f659-44c5-a352-4a0190c13b29 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:30.678828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4195-f659-44c5-a352-4a0190c13b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:30.754434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6faa4195-f659-44c5-a352-4a0190c13b29 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:30.755673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6faa4195-f659-44c5-a352-4a0190c13b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:30.832835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:30.855981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 675/1336] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:30.896736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:30.898292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:31.011594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83040c-47b0-4dc0-8b87-5cf3d1f1e13b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:31.043135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:31.090645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:31.091291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:31.091571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:31.091844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:31.093501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:31.199392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:31.200818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e64e74-ac8a-4be1-bb5b-cb1e3742cc3b service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:31.219380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 662/1337] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:34.044391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:34.044940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:34.044940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:34.046370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 676/1338] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:34.051574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290533-2362-4a73-b4c3-ee1b37f4e8b9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:34.051994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290533-2362-4a73-b4c3-ee1b37f4e8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:34.052129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290533-2362-4a73-b4c3-ee1b37f4e8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:34.117833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-69290533-2362-4a73-b4c3-ee1b37f4e8b9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:34.119050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-69290533-2362-4a73-b4c3-ee1b37f4e8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:34.127276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:34.216323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 663/1339] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:34.252240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:34.253810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:34.356786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c24b9-a9a8-4add-a0a7-f85dacc02184 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:34.430594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:34.431242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:34.431533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:34.431813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:34.433471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:34.468428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:34.541630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:34.543093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3658-f3ad-4281-abb8-e3c413e38eee service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:34.562898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 677/1340] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:38.976328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:38.976880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:38.976880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:38.978238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 664/1341] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:38.983883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5a1e-6993-48ad-96b3-9c976124e92e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:38.984390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5a1e-6993-48ad-96b3-9c976124e92e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:38.984487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5a1e-6993-48ad-96b3-9c976124e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:39.053476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bb1e5a1e-6993-48ad-96b3-9c976124e92e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:39.054966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bb1e5a1e-6993-48ad-96b3-9c976124e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:39.159849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:39.171354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 678/1342] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:39.210529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:39.212036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:39.330343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d713a-1b61-4801-9056-a791111ee1a4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:39.368956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:39.405457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:39.406145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:39.406460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:39.406741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:39.408405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:39.521465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:39.522947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7a7c-185f-46dc-ade6-0a82de2771e8 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:39.542583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 665/1343] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:42.717826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:42.718343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:42.718343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:42.719677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 679/1344] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:42.725111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de1ac-9fb7-420b-87df-5a550669d304 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:42.725566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de1ac-9fb7-420b-87df-5a550669d304 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:42.725659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de1ac-9fb7-420b-87df-5a550669d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:42.793279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b58de1ac-9fb7-420b-87df-5a550669d304 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:42.794558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b58de1ac-9fb7-420b-87df-5a550669d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:42.814698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:42.899077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 666/1345] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:42.936247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:42.937799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:43.043598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68da264-2912-43d1-b62f-7eb05a5e69f6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:43.119101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:43.119784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:43.120073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:43.120358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:43.122177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:43.169881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:43.237705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:43.239181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410b1ad-98c0-4fb0-a339-72553c8cf68c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:43.258464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 680/1346] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:47.772726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:47.773359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:47.773359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:47.775988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 667/1347] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:22:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:22:47.780040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e76-370a-4424-9143-866e32c131b0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:47.780498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e76-370a-4424-9143-866e32c131b0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:47.780610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e76-370a-4424-9143-866e32c131b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:47.859171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-66942e76-370a-4424-9143-866e32c131b0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:47.860384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-66942e76-370a-4424-9143-866e32c131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:47.964752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:47.967984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 681/1348] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:48.006532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:22:48.008082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:48.121575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e34c66-e081-4440-a6c8-889081c3d45a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:22:48.170988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:48.200743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:22:48.201502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:48.201818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:48.202102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:48.203808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:48.315458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:22:48.316917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b16d1-d1eb-4aa8-a92d-827e61082382 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:22:48.336396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 668/1349] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:22:53.400436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45add-2645-4511-bd91-88902395d8d2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:53.400972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45add-2645-4511-bd91-88902395d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:53.400972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45add-2645-4511-bd91-88902395d8d2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:53.402839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 682/1350] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:53.407964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc211-830f-49f8-a554-e0f1e0ce270a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:53.410776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc211-830f-49f8-a554-e0f1e0ce270a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:53.410776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc211-830f-49f8-a554-e0f1e0ce270a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:53.551759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-07dfc211-830f-49f8-a554-e0f1e0ce270a None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:53.553792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-07dfc211-830f-49f8-a554-e0f1e0ce270a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:53.576272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:53.695673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 669/1351] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:22:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:22:57.946056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6be88-198b-4621-8a06-82a86d1bde4a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:57.946610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6be88-198b-4621-8a06-82a86d1bde4a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:57.946610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6be88-198b-4621-8a06-82a86d1bde4a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:57.948237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 683/1352] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:57.954155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3f56-c56e-49da-b7f7-8cc2e0e8b0c4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:57.954583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3f56-c56e-49da-b7f7-8cc2e0e8b0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:57.955609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3f56-c56e-49da-b7f7-8cc2e0e8b0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:58.041195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0a8b3f56-c56e-49da-b7f7-8cc2e0e8b0c4 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:58.042484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0a8b3f56-c56e-49da-b7f7-8cc2e0e8b0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:58.132583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:22:58.171306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 670/1353] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:22:59.545070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592f11-8212-4eac-9ae4-248610e0aedc None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:59.545574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592f11-8212-4eac-9ae4-248610e0aedc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:59.545756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592f11-8212-4eac-9ae4-248610e0aedc None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:59.547580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 684/1354] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:22:59.552457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34207508-030a-4ec7-bb68-0dade8bf7b7d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:22:59.552885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34207508-030a-4ec7-bb68-0dade8bf7b7d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:22:59.553152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34207508-030a-4ec7-bb68-0dade8bf7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:22:59.628670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-34207508-030a-4ec7-bb68-0dade8bf7b7d None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:22:59.630454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34207508-030a-4ec7-bb68-0dade8bf7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:22:59.743247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 671/1355] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:03.214491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329610-1ff9-439c-83db-c45f6eb4c954 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:03.214987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329610-1ff9-439c-83db-c45f6eb4c954 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:03.214987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329610-1ff9-439c-83db-c45f6eb4c954 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:03.216331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 685/1356] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:03.221699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ad4f-adab-4a48-8116-b7632bc0c038 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:03.222139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ad4f-adab-4a48-8116-b7632bc0c038 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:03.222342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ad4f-adab-4a48-8116-b7632bc0c038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:03.260825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:03.294075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bd01ad4f-adab-4a48-8116-b7632bc0c038 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:03.295192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bd01ad4f-adab-4a48-8116-b7632bc0c038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:03.397062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 672/1357] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:03.450669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:03.451191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:03.451647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:03.453509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 686/1358] 10.222.0.93 () {54 vars in 961 bytes} [Thu Jul 14 14:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:03.462632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fc1f-0b2c-4713-8456-533343c9550c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:03.463705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fc1f-0b2c-4713-8456-533343c9550c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:03.464145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fc1f-0b2c-4713-8456-533343c9550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:03.554318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b274fc1f-0b2c-4713-8456-533343c9550c None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:03.554318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b274fc1f-0b2c-4713-8456-533343c9550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:03.609101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:03.684766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 673/1359] 10.222.0.93 () {58 vars in 1058 bytes} [Thu Jul 14 14:23:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:03.694267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:03.695802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:03.849923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cafd6-027c-4c4c-85dd-ffc721a5e686 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:03.928388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:03.975113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:03.976035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:03.976426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:03.976828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:03.979000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:04.083984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:04.085608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1aee0-47fa-47ab-b99a-920c9fed0303 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:04.104473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 687/1360] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:23:06.417092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:06.417649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:06.417649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:06.419128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 674/1361] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:06.424599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4814-6285-4b6b-9e68-ce883308080f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:06.425075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4814-6285-4b6b-9e68-ce883308080f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:06.425160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4814-6285-4b6b-9e68-ce883308080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:06.505686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-07ca4814-6285-4b6b-9e68-ce883308080f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:06.507101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-07ca4814-6285-4b6b-9e68-ce883308080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:06.549557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:06.621131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 688/1362] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:06.667152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:06.668819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:06.842075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8770c-f676-4ea2-8b6e-99f55f2f24ad None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:06.852768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:06.999931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:07.000756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:07.001111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:07.001455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:07.003368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:07.103995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:07.105603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28738271-a738-4467-bd51-84f2f2edec1e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:07.125266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 675/1363] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:09.644187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df669e-d626-41b2-8fdb-baba91d052f2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:09.644777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df669e-d626-41b2-8fdb-baba91d052f2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:09.644777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df669e-d626-41b2-8fdb-baba91d052f2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:09.646202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 689/1364] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:09.651941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b752e4-1e90-4470-814c-5a19d1d1939c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:09.652405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b752e4-1e90-4470-814c-5a19d1d1939c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:09.652489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b752e4-1e90-4470-814c-5a19d1d1939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:09.670723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:09.724322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e4b752e4-1e90-4470-814c-5a19d1d1939c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:09.725720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e4b752e4-1e90-4470-814c-5a19d1d1939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:09.833949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 676/1365] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:09.881528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:09.882110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:09.882395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:09.883768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 690/1366] 10.222.0.93 () {54 vars in 961 bytes} [Thu Jul 14 14:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:09.889683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba6ad-00ca-460b-b3c9-c4bfc67f5a28 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:09.890079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba6ad-00ca-460b-b3c9-c4bfc67f5a28 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:09.890412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba6ad-00ca-460b-b3c9-c4bfc67f5a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:09.971195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-aaeba6ad-00ca-460b-b3c9-c4bfc67f5a28 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:09.972481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aaeba6ad-00ca-460b-b3c9-c4bfc67f5a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:10.011862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:10.115021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 677/1367] 10.222.0.93 () {58 vars in 1058 bytes} [Thu Jul 14 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:10.122703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:10.124343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:10.279207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acaf68-bd77-494b-87d6-dd366151300d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:10.300852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:10.397939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:10.398511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:10.398795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:10.399071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:10.400705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:10.496441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:10.498074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9f548-f43e-4438-a3eb-3a78ce9050fc service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:10.518150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 691/1368] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:11.529629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35c27-4ac8-4182-9ed8-d83cb66dca89 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:11.530292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35c27-4ac8-4182-9ed8-d83cb66dca89 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:11.530292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35c27-4ac8-4182-9ed8-d83cb66dca89 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:11.531778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 678/1369] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:11.537620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2eb2-3541-403d-8f99-edfed5188547 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:11.538036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2eb2-3541-403d-8f99-edfed5188547 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:11.538222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2eb2-3541-403d-8f99-edfed5188547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:11.569200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:11.605374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-55fd2eb2-3541-403d-8f99-edfed5188547 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:11.606618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-55fd2eb2-3541-403d-8f99-edfed5188547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:11.711185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 692/1370] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:12.057321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:12.057321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:12.058190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:12.059227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 679/1371] 10.222.0.93 () {54 vars in 963 bytes} [Thu Jul 14 14:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:12.066427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277242dc-bbef-494e-94b2-bd8143833e6b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:12.066427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277242dc-bbef-494e-94b2-bd8143833e6b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:12.066427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277242dc-bbef-494e-94b2-bd8143833e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:12.159538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-277242dc-bbef-494e-94b2-bd8143833e6b None None] MFA Rules not processed for user `398d1954cb704f3898d3984f5c6102a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:12.161284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-277242dc-bbef-494e-94b2-bd8143833e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:12.208957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:12.240014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 693/1372] 10.222.0.93 () {58 vars in 1060 bytes} [Thu Jul 14 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:12.247291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:12.248840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:12.360629 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b180-dafd-448f-a596-42edb7df0d4b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:12.403590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:12.438806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:12.439919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:12.440570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:12.441185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:12.443354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:12.567631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:12.569267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad43d5-79ca-4996-8d44-3ddeeb8997dc service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:12.589398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 680/1373] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:12.843316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:12.843946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:12.844553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:12.847160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 694/1374] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:12.851842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdc7f-1812-47f9-893a-5c3c190f0ba0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:12.852187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdc7f-1812-47f9-893a-5c3c190f0ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:12.852416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdc7f-1812-47f9-893a-5c3c190f0ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:12.927027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-982fdc7f-1812-47f9-893a-5c3c190f0ba0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:12.933220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-982fdc7f-1812-47f9-893a-5c3c190f0ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:12.960181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:13.059038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 681/1375] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:13.103344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:13.104944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:13.256551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60f02e-0022-417d-837c-254c0b03daf0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:13.312805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:13.380075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:13.380737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:13.381022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:13.381303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:13.382993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:13.472701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:13.474326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b297d-10cf-4de4-a2bb-54d47ae44383 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:13.492927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 695/1376] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:15.757974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:15.758509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:15.758509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:15.760203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 682/1377] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:15.765910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22881a-738d-436d-95ae-a426e61624a6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:15.766367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22881a-738d-436d-95ae-a426e61624a6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:15.766460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22881a-738d-436d-95ae-a426e61624a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:15.833680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3c22881a-738d-436d-95ae-a426e61624a6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:15.835028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3c22881a-738d-436d-95ae-a426e61624a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:15.841799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:15.933397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 696/1378] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:15.977597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:15.979145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:16.167356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a3ea8-9556-4534-9cda-7e448b40351d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:16.171164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:16.323911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:16.324732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:16.325113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:16.325507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:16.327716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:16.425018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:16.427077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89cb3f-1fe3-4db6-82c8-a16d6290f03b service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:16.446855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 683/1379] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:18.687553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:18.688101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:18.688101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:18.689490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 697/1380] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:18.694951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df60dd-711c-4d6d-8b93-2c7a148b710f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:18.695390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df60dd-711c-4d6d-8b93-2c7a148b710f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:18.695520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df60dd-711c-4d6d-8b93-2c7a148b710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:18.706008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:18.775094 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-95df60dd-711c-4d6d-8b93-2c7a148b710f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:18.776311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95df60dd-711c-4d6d-8b93-2c7a148b710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:18.882287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 684/1381] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:18.928240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:18.929994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:19.091416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:19.103620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4afb7-ed09-4c64-a484-d2f042e7cbcb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:19.230502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:19.231215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:19.231495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:19.231777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:19.233603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:19.354470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:19.356108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3cf58-092d-4201-939f-7834ba31a17d service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:19.368392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:19.380926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 698/1382] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:21.958840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:21.959357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:21.959357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:21.960636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 685/1383] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:21.966568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b541268-ce5e-424c-8c45-87e5caababe9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:21.967021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b541268-ce5e-424c-8c45-87e5caababe9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:21.967115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b541268-ce5e-424c-8c45-87e5caababe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:22.045893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2b541268-ce5e-424c-8c45-87e5caababe9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:22.047469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b541268-ce5e-424c-8c45-87e5caababe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:22.157923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 699/1384] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:22.202512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:22.204053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:22.295429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:22.361791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413d258-d658-4d3e-be62-4ca5ed7bff0b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:22.489087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:22.489911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:22.490273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:22.490621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:22.492784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:22.547525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:22.591848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:22.593684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdee0e3-1aa4-4fb5-a8da-d44da2a9bc65 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:22.613938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 686/1385] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:24.733393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:24.734033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:24.734033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:24.735534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 700/1386] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:24.741481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c4db0-09d8-4667-880b-51066cbec08f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:24.741996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c4db0-09d8-4667-880b-51066cbec08f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:24.742184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c4db0-09d8-4667-880b-51066cbec08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:24.828559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b38c4db0-09d8-4667-880b-51066cbec08f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:24.829901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b38c4db0-09d8-4667-880b-51066cbec08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:24.931907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 687/1387] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:24.976924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:24.978483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:25.059510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:25.144328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7a03-6e1e-4266-997b-0df0789581fa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:25.290527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:25.291191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:25.291478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:25.291763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:25.293443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:25.333860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:25.398919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:25.401130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f7c0-ac3f-4dca-89ce-93cd9f2b7e69 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:25.425406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 701/1388] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:27.608977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:27.609542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:27.609656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:27.610992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 688/1389] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:27.616655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940fe5-28f9-41c4-8f38-3fcb91cd6901 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:27.617069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940fe5-28f9-41c4-8f38-3fcb91cd6901 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:27.617217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940fe5-28f9-41c4-8f38-3fcb91cd6901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:27.685300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ce940fe5-28f9-41c4-8f38-3fcb91cd6901 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:27.686693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce940fe5-28f9-41c4-8f38-3fcb91cd6901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:27.786614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 702/1390] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:27.830080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:27.831607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:27.863307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:27.978530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f12d5-3acd-419b-bdac-92da654c1939 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:28.087910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:28.099425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:28.102282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.102282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.102282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.102562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1417-5e1a-4c9e-8636-8fd3792dc0bb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.102562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1417-5e1a-4c9e-8636-8fd3792dc0bb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.102562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1417-5e1a-4c9e-8636-8fd3792dc0bb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.102824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:28.103412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 703/1391] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:23:28.108331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aef80-75be-4c6b-94ae-3967d8eaf5bc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.108614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aef80-75be-4c6b-94ae-3967d8eaf5bc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.108892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aef80-75be-4c6b-94ae-3967d8eaf5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.123338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:28.180234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-661aef80-75be-4c6b-94ae-3967d8eaf5bc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:28.181487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-661aef80-75be-4c6b-94ae-3967d8eaf5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:28.202135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:28.203779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7473ec-cd6a-4495-9f35-33facb482661 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:28.222194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 689/1392] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:28.279931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 704/1393] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:28.336817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.340037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.340037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.340037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 690/1394] 10.222.0.93 () {54 vars in 962 bytes} [Thu Jul 14 14:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:28.344481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61717-5545-42a0-87f2-51d72b60ad93 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.344801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61717-5545-42a0-87f2-51d72b60ad93 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.345084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61717-5545-42a0-87f2-51d72b60ad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.418088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f9d61717-5545-42a0-87f2-51d72b60ad93 None None] MFA Rules not processed for user `1235df33fd6f4917850cf0f2aa850b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:28.419283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f9d61717-5545-42a0-87f2-51d72b60ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:28.455468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:28.504758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 705/1395] 10.222.0.93 () {58 vars in 1059 bytes} [Thu Jul 14 14:23:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:28.512742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:28.514348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:28.627875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b4aff-c92a-4d09-a509-808ce6759a11 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:28.641290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:28.702825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:28.703476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:28.703757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:28.704031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:28.705661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:28.819933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:28.821375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761833e3-11f7-4294-8d9f-1ec20065db3e service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:28.839138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 691/1396] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:29.794605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760971d-c674-4c9d-842c-ef96e8261cb4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:29.795158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760971d-c674-4c9d-842c-ef96e8261cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:29.795158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760971d-c674-4c9d-842c-ef96e8261cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:29.863404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2760971d-c674-4c9d-842c-ef96e8261cb4 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:29.864542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2760971d-c674-4c9d-842c-ef96e8261cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:29.954900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 706/1397] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:23:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:29.997829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:29.998338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:29.998510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:29.999998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 692/1398] 10.222.0.93 () {54 vars in 942 bytes} [Thu Jul 14 14:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:30.007002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfaabf-522d-4c26-b085-3d32de3bf891 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:30.007300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfaabf-522d-4c26-b085-3d32de3bf891 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:30.007585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfaabf-522d-4c26-b085-3d32de3bf891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:30.067957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:30.088069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2fbfaabf-522d-4c26-b085-3d32de3bf891 None None] MFA Rules not processed for user `4d4c712ae2b04fc68eed213213452664`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:30.096244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2fbfaabf-522d-4c26-b085-3d32de3bf891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:30.202670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 707/1399] 10.222.0.93 () {58 vars in 1039 bytes} [Thu Jul 14 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:30.212061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:30.213633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:30.259264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:30.350873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088445-fb5a-4505-9935-a405d8b5ac9d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:30.458879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:30.459363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:30.459655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:30.459933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:30.461618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:30.493282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:30.509540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226fa-3ee2-4c35-b5fa-8bb7428c817f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:30.509861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226fa-3ee2-4c35-b5fa-8bb7428c817f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:30.510153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226fa-3ee2-4c35-b5fa-8bb7428c817f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:30.511544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 708/1400] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:30.516558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd014-52c2-42bd-b5a7-3856e05aa4c9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:30.516851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd014-52c2-42bd-b5a7-3856e05aa4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:30.517174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd014-52c2-42bd-b5a7-3856e05aa4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:30.549034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:30.550791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:30.558826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 693/1401] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:23:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:23:30.586699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6b3bd014-52c2-42bd-b5a7-3856e05aa4c9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:30.587927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6b3bd014-52c2-42bd-b5a7-3856e05aa4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:30.644982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:30.685813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 709/1402] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:30.730840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:30.732066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:30.825035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1966c-398a-4095-9e21-afd7e47b732b service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:30.895285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:30.954336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:30.955123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:30.955418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:30.955700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:30.958148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:31.059821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:31.062302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ca316-f472-4d35-94cf-b955e412f958 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:31.083434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 694/1403] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:33.369385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:33.369385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:33.369385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:33.369385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 710/1404] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:33.374543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9cf3-f6cb-49b6-bd1f-d9a415cf7d5c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:33.374967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9cf3-f6cb-49b6-bd1f-d9a415cf7d5c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:33.375140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9cf3-f6cb-49b6-bd1f-d9a415cf7d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:33.456216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9e6b9cf3-f6cb-49b6-bd1f-d9a415cf7d5c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:33.457796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9e6b9cf3-f6cb-49b6-bd1f-d9a415cf7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:33.496521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:33.565025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 695/1405] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:33.609102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:33.610820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:33.783736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773198fe-3deb-45a5-8854-38162c37d573 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:33.834030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:33.926073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:33.926735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:33.927026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:33.927306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:33.929037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:34.033371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:34.035116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715fc10-3b3f-4e24-a030-c02155127f6a service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:34.054438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 711/1406] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:36.370519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:36.371067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:36.371067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:36.372344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 696/1407] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:36.380340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569718-4286-436f-9444-3517f0a9d58e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:36.380340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569718-4286-436f-9444-3517f0a9d58e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:36.380340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569718-4286-436f-9444-3517f0a9d58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:36.468221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:36.475954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a6569718-4286-436f-9444-3517f0a9d58e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:36.479129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a6569718-4286-436f-9444-3517f0a9d58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:36.587874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 712/1408] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:23:36.633866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:36.635471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:36.814895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767af2b-f3db-44b8-b150-12b661097318 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:36.834188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:36.950414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:36.951251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:36.951600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:36.951975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:36.953725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:37.060733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:37.062496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7c07b-3b95-4b17-a5c2-0dc25893641d service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:37.090093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 697/1409] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:46.585613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=44 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:47.085281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:54.928483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456deaf-8991-4dcf-a84b-9b33369f46dc None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:54.929076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456deaf-8991-4dcf-a84b-9b33369f46dc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:54.929076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456deaf-8991-4dcf-a84b-9b33369f46dc None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:54.930695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 713/1410] 10.222.0.93 () {54 vars in 918 bytes} [Thu Jul 14 14:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:54.936008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d220-2652-4bb3-ae20-7df942d776e7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:54.936450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d220-2652-4bb3-ae20-7df942d776e7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:54.936597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d220-2652-4bb3-ae20-7df942d776e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:55.015446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a461d220-2652-4bb3-ae20-7df942d776e7 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:55.016675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a461d220-2652-4bb3-ae20-7df942d776e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:55.130600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 698/1411] 10.222.0.93 () {58 vars in 1015 bytes} [Thu Jul 14 14:23:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:55.145871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb4a9f-8118-4a8a-877a-0e5e80f80c8b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:55.146191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb4a9f-8118-4a8a-877a-0e5e80f80c8b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:55.146472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb4a9f-8118-4a8a-877a-0e5e80f80c8b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:55.147828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 714/1412] 10.222.0.93 () {54 vars in 918 bytes} [Thu Jul 14 14:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:55.152431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a593-3c28-4051-9230-85f4a3b00e74 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:55.152722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a593-3c28-4051-9230-85f4a3b00e74 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:55.153000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a593-3c28-4051-9230-85f4a3b00e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:55.224920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:55.227463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-96d1a593-3c28-4051-9230-85f4a3b00e74 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:55.228682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-96d1a593-3c28-4051-9230-85f4a3b00e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:55.351542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 699/1413] 10.222.0.93 () {58 vars in 1015 bytes} [Thu Jul 14 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:57.927196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:57.927738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:57.927738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:57.929181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:57.934308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f14d-48b6-4c58-a6b0-50b3ac4d1684 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:57.934717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f14d-48b6-4c58-a6b0-50b3ac4d1684 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:57.934868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f14d-48b6-4c58-a6b0-50b3ac4d1684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:57.998801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1a70f14d-48b6-4c58-a6b0-50b3ac4d1684 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:58.000001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a70f14d-48b6-4c58-a6b0-50b3ac4d1684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:58.055708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:58.104319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:58.147563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:58.149120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:58.293279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac4dc6-208a-4034-9240-f1d96423d2a1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:23:58.365706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:58.408655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:23:58.410145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:58.410145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:58.410145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:58.412417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:58.504975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:23:58.506601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99167749-e739-47e6-85b8-61dafba9f7c3 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:23:58.524481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 716/1416] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:23:59.425659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:59.426300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:59.426300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:59.427630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:23:59.434598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e67d-33e3-49ad-b0e2-ecdc9b6c5d3c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:23:59.435087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e67d-33e3-49ad-b0e2-ecdc9b6c5d3c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:23:59.435297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e67d-33e3-49ad-b0e2-ecdc9b6c5d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:23:59.511261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cde1e67d-33e3-49ad-b0e2-ecdc9b6c5d3c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:23:59.512502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cde1e67d-33e3-49ad-b0e2-ecdc9b6c5d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:59.539417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:23:59.610181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:23:59.760927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:23:59.762625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:23:59.913480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a5de-1f36-46f0-a0a9-32bb0b4bb63b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:00.070406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:00.145173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:00.145868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:00.146319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:00.146601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:00.148303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:00.243584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:00.245203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a1cb0-f737-4911-8b37-a9d412105415 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:00.265093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 702/1419] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:01.159303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:01.159902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:01.159902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:01.161251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:01.166838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2424-1a0c-498e-8836-9caf9da4e7ad None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:01.167305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2424-1a0c-498e-8836-9caf9da4e7ad None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:01.167475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2424-1a0c-498e-8836-9caf9da4e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:01.220108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:01.241836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-376c2424-1a0c-498e-8836-9caf9da4e7ad None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:01.243053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-376c2424-1a0c-498e-8836-9caf9da4e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:01.338090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:01.490035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:01.491556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:01.638098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdd154-83a9-42f8-a50c-6e73ef15b808 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:01.650048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:01.750629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:01.751280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:01.751567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:01.751849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:01.753518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:01.840746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:01.842375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59e0bf-78c0-4903-b2f5-ccf3d11f94e4 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:01.860681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 719/1422] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:02.725986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:02.726514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:02.726618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:02.728000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:02.733277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf942-01c6-401b-95af-5bc40fe464cd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:02.733727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf942-01c6-401b-95af-5bc40fe464cd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:02.733860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf942-01c6-401b-95af-5bc40fe464cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:02.754019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:02.812413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bd6cf942-01c6-401b-95af-5bc40fe464cd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:02.814424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bd6cf942-01c6-401b-95af-5bc40fe464cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:02.911939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:03.062439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:03.064004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:03.172508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:03.208880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5ae26-5deb-40e7-8f50-d600bea64ebb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:03.321087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:03.321739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:03.322061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:03.322338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:03.323983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:03.399303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:03.412444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:03.414075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703a866-78b7-4c94-a195-2aef7d821cb8 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:03.432488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 705/1425] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:04.303292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:04.303894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:04.303894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:04.305232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:04.310300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad100-8457-480f-a0d9-217e7db60d99 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:04.310749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad100-8457-480f-a0d9-217e7db60d99 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:04.311860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad100-8457-480f-a0d9-217e7db60d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:04.387993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-824ad100-8457-480f-a0d9-217e7db60d99 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:04.389263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-824ad100-8457-480f-a0d9-217e7db60d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:04.483112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:04.648541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:04.650025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:04.734226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:04.819817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0c6d-296f-47c5-bcdb-76f9c72d2f85 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:04.934893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:04.935462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:04.935748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:04.936028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:04.937673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:04.986129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:05.027097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:05.028850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce761eb9-c6b0-4ae5-a0cf-a66db4329c1c service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:05.047338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 722/1428] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:05.933978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25add7a-8303-4500-8099-0400b378c212 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:05.934721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25add7a-8303-4500-8099-0400b378c212 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:05.934721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25add7a-8303-4500-8099-0400b378c212 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:05.935835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:05.941327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c81d8-95e1-453b-930e-7f96422c7319 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:05.941822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c81d8-95e1-453b-930e-7f96422c7319 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:05.941940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c81d8-95e1-453b-930e-7f96422c7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:06.016981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8c2c81d8-95e1-453b-930e-7f96422c7319 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:06.018257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8c2c81d8-95e1-453b-930e-7f96422c7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:06.120850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:06.271921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25add7a-8303-4500-8099-0400b378c212 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:06.273445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d25add7a-8303-4500-8099-0400b378c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:06.321690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:06.420376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25add7a-8303-4500-8099-0400b378c212 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:06.550098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:06.550663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:06.550945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:06.551218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:06.552866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:06.559866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:06.641803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:06.643444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1ae2d-1a95-45af-8e67-e996815e4fdb service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:06.661433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 708/1431] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:07.525421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:07.526012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:07.526012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:07.527414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:07.532402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad9d-b1de-4a9b-be3d-446fd7bf5519 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:07.532882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad9d-b1de-4a9b-be3d-446fd7bf5519 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:07.532993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad9d-b1de-4a9b-be3d-446fd7bf5519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:07.608533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6bdbad9d-b1de-4a9b-be3d-446fd7bf5519 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:07.609744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6bdbad9d-b1de-4a9b-be3d-446fd7bf5519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:07.676377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:07.705864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:07.857858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:07.859307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:07.878902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:08.008491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3f7f1-c470-4c3d-9800-85cb292f2915 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:08.095981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:08.122323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:08.122907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:08.123199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:08.123480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:08.125129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:08.216102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:08.217689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4237-099e-471a-9537-84c1cb3af449 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:08.235995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 725/1434] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:09.121925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:09.122465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:09.122465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:09.123880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:09.129394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796bed-723b-40e4-a54d-16579223d9ff None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:09.129842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796bed-723b-40e4-a54d-16579223d9ff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:09.130081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796bed-723b-40e4-a54d-16579223d9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:09.204516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4f796bed-723b-40e4-a54d-16579223d9ff None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:09.206302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4f796bed-723b-40e4-a54d-16579223d9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:09.248599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:09.303536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:09.454492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:09.455944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:09.601684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0c009-e444-4850-bf17-01213cedcb93 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:09.656412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:09.715027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:09.715603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:09.715888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:09.716169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:09.717909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:09.806708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:09.808315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4cd96-975f-49d3-bb5c-7162f597b9f0 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:09.826081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 711/1437] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:10.688717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:10.690549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:10.690549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:10.691263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:10.696917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b2d0e-dfd2-4ffa-bbcc-6ac7fd9b5f15 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:10.697386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b2d0e-dfd2-4ffa-bbcc-6ac7fd9b5f15 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:10.697486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b2d0e-dfd2-4ffa-bbcc-6ac7fd9b5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:10.776894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4b6b2d0e-dfd2-4ffa-bbcc-6ac7fd9b5f15 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:10.778338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4b6b2d0e-dfd2-4ffa-bbcc-6ac7fd9b5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:10.780349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:10.883469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:11.035502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:11.037018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:11.202088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695c8b8-225d-4de8-be12-771e79cba65d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:11.229415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:11.317226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:11.317813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:11.318101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:11.318380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:11.320027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:11.407094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:11.409213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835ced0-0fdc-404f-a461-93f57f5899bf service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:11.427357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 728/1440] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:13.388265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:13.388800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:13.388800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:13.390196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:13.396011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd495-2b56-49f4-985a-eb86cf42f0b4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:13.396449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd495-2b56-49f4-985a-eb86cf42f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:13.396575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd495-2b56-49f4-985a-eb86cf42f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:13.436122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:13.460952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e9bdd495-2b56-49f4-985a-eb86cf42f0b4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:13.462223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e9bdd495-2b56-49f4-985a-eb86cf42f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:13.559893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:13.600281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:13.601916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:13.709427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8214c-3bbd-4924-aace-af7598b80c1e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:13.742204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:13.783932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:13.784581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:13.784869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:13.785149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:13.786850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:13.896020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:13.897449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3def-e429-4ec8-8665-1218ca9b6ce2 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:13.918359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 714/1443] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:16.124503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:16.125033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:16.125160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:16.126648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:16.132016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f162a-30ea-4571-8d56-8270e9109181 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:16.132473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f162a-30ea-4571-8d56-8270e9109181 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:16.132604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f162a-30ea-4571-8d56-8270e9109181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:16.197349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0c9f162a-30ea-4571-8d56-8270e9109181 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:16.198595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0c9f162a-30ea-4571-8d56-8270e9109181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:16.211843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:16.296350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:16.335184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:16.336746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:16.438646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dba83-66e5-438c-b714-fe4cc97180f9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:16.450685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:16.511636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:16.512279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:16.512561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:16.512837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:16.514494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:16.633962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:16.635296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4f11f-8b66-4f58-a2c1-3a82f93c1278 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:16.654662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 731/1446] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:19.086644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:19.087167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:19.087288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:19.088568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:19.094099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba4c4-efe3-4341-b89f-d705e4fbd627 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:19.094511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba4c4-efe3-4341-b89f-d705e4fbd627 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:19.094657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba4c4-efe3-4341-b89f-d705e4fbd627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:19.141435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:19.160006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-195ba4c4-efe3-4341-b89f-d705e4fbd627 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:19.161229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-195ba4c4-efe3-4341-b89f-d705e4fbd627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:19.258649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:19.297697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:19.299777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:19.403180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce5263-460b-4378-8384-ae469e8d21c9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:19.444870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:19.478986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:19.479632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:19.479913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:19.480184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:19.481849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:19.588840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:19.590322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3523f-8ac3-46fc-a84d-63665a4e48e4 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:19.609083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 717/1449] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:21.804471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:21.805009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:21.805009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:21.806453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:21.811902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec863bc-788f-4272-8f0a-ac4d7fda255c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:21.812327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec863bc-788f-4272-8f0a-ac4d7fda255c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:21.812491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec863bc-788f-4272-8f0a-ac4d7fda255c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:21.877028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6ec863bc-788f-4272-8f0a-ac4d7fda255c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:21.878208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6ec863bc-788f-4272-8f0a-ac4d7fda255c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:21.899215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:21.978120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:22.020051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:22.021580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:22.128794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd639a-1310-4168-97de-4afe4a387c7d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:22.149012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:22.322757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:22.323421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:22.323707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:22.323984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:22.325688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:22.435932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:22.437273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d54d5-384b-4f95-a6ba-f2499cf3d12f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:22.455555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 734/1452] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:24.539029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:24.539595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:24.539595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:24.540957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:24.546186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc61a-8421-4be4-bd3b-0bab6927fc13 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:24.546631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc61a-8421-4be4-bd3b-0bab6927fc13 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:24.546751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc61a-8421-4be4-bd3b-0bab6927fc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:24.607813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:24.611536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5aecc61a-8421-4be4-bd3b-0bab6927fc13 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:24.612753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5aecc61a-8421-4be4-bd3b-0bab6927fc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:24.711407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:24.749510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:24.751099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:24.871168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e4d41-8d0f-4dcd-9796-6216402bd7a0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:24.920475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:24.944721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:24.945371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:24.945658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:24.945967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:24.947614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:25.057574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:25.059401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb54e59-e6c4-45ce-af77-5b936af6901e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:25.078103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 720/1455] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:27.203055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:27.203632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:27.203632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:27.205606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:27.211080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fe46a-0342-469c-b55f-c6413186c138 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:27.211518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fe46a-0342-469c-b55f-c6413186c138 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:27.213105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fe46a-0342-469c-b55f-c6413186c138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:27.276447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ea4fe46a-0342-469c-b55f-c6413186c138 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:27.277679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ea4fe46a-0342-469c-b55f-c6413186c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:27.306119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:27.372900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:27.411267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:27.412826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:27.515305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1841a-f013-4fcf-818a-d954aa4f7bd4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:27.543342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:27.589463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:27.590286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:27.590455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:27.590927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:27.592368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:27.701531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:27.702923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b90c0-0430-4a13-85b5-7b4273ff0402 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:27.721292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 737/1458] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:29.837447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:29.837996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:29.837996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:29.839342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:29.844795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831384c0-5ed7-46a6-be30-75e808194a37 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:29.845864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831384c0-5ed7-46a6-be30-75e808194a37 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:29.845986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831384c0-5ed7-46a6-be30-75e808194a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:29.909366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-831384c0-5ed7-46a6-be30-75e808194a37 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:29.910605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-831384c0-5ed7-46a6-be30-75e808194a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:29.918810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:30.006117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:30.043658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:30.045193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:30.149708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b7c0c-4f26-4e98-b0b7-1c5d5c647d1b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:30.205319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:30.223600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:30.224249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:30.224536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:30.224815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:30.226502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:30.335537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:30.336981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a800598-0117-4fad-b9a3-bd28fce0a23f service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:30.355970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 723/1461] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:32.431951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:32.432490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:32.432490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:32.433840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:32.439101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df989759-7ffa-4083-9085-73b0599793ff None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:32.439538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df989759-7ffa-4083-9085-73b0599793ff None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:32.439657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df989759-7ffa-4083-9085-73b0599793ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:32.503951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-df989759-7ffa-4083-9085-73b0599793ff None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:32.505163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-df989759-7ffa-4083-9085-73b0599793ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:32.545307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:32.613449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:32.651006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:32.652555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:32.765620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa27e6-1203-4003-9a95-e895679c16ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:32.806285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:32.844913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:32.846125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:32.846416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:32.846693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:32.848389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:32.967018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:32.968510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6cc3f-3055-47b2-b305-fa1c39a713ca service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:32.990049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 740/1464] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:35.050065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:35.050667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:35.050667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:35.052032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:35.057389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64bf-a1fd-4d58-b24d-4f79c98ab666 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:35.057806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64bf-a1fd-4d58-b24d-4f79c98ab666 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:35.058004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64bf-a1fd-4d58-b24d-4f79c98ab666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:35.148707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-da8c64bf-a1fd-4d58-b24d-4f79c98ab666 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:35.148707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-da8c64bf-a1fd-4d58-b24d-4f79c98ab666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:35.149107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:35.227165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:35.264607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:35.266238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:35.371425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542d2c-557e-41d5-bc69-5b25fc01d32f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:35.437049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:35.443771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:35.447491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:35.447491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:35.447491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:35.447491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:35.556148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:35.557580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0cb37-6792-4b0e-8e4d-4c50c25d192a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:35.575835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 726/1467] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:37.640853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:37.641448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:37.641448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:37.643404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:37.648760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd6-3989-412e-902a-f0cbfd907051 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:37.649199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd6-3989-412e-902a-f0cbfd907051 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:37.649308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd6-3989-412e-902a-f0cbfd907051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:37.713962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fa7ebcd6-3989-412e-902a-f0cbfd907051 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:37.715189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fa7ebcd6-3989-412e-902a-f0cbfd907051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:37.763025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:37.811582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:37.847712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:37.849221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:37.980122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baac71-0d9c-44a1-a836-c625b2840eee None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:38.027909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:38.055455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:38.056242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:38.056532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:38.056806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:38.058611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:38.169564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:38.170948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6941d3e-5afc-42f5-a4cd-95db12c528f2 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:38.190085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 743/1470] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:24:47.810041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=22 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:48.187259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=66 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:58.865581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec997-1a5b-4fca-a4b7-2a84c292c315 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:58.866201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec997-1a5b-4fca-a4b7-2a84c292c315 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:58.866201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ec997-1a5b-4fca-a4b7-2a84c292c315 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:58.867587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 728/1471] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:58.872896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06ff15-ab7f-425f-a47d-85eac48cb972 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:58.873508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06ff15-ab7f-425f-a47d-85eac48cb972 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:58.873508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06ff15-ab7f-425f-a47d-85eac48cb972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:58.944951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5f06ff15-ab7f-425f-a47d-85eac48cb972 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:58.946221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5f06ff15-ab7f-425f-a47d-85eac48cb972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:59.052049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 744/1472] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:59.143183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:59.143472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:59.143748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:59.145136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 729/1473] 10.222.0.93 () {54 vars in 962 bytes} [Thu Jul 14 14:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:24:59.150033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe770e-ecd0-4d41-8e92-c87ecb9d4b4c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:59.150324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe770e-ecd0-4d41-8e92-c87ecb9d4b4c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:59.150601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe770e-ecd0-4d41-8e92-c87ecb9d4b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:59.254375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-edfe770e-ecd0-4d41-8e92-c87ecb9d4b4c None None] MFA Rules not processed for user `1235df33fd6f4917850cf0f2aa850b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:24:59.255659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-edfe770e-ecd0-4d41-8e92-c87ecb9d4b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:59.279909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:59.339153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 745/1474] 10.222.0.93 () {58 vars in 1059 bytes} [Thu Jul 14 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:24:59.346875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:24:59.349408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:59.456473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96698-5bc3-433e-98b5-70a4c95a41ba None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:24:59.539587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:24:59.540253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:24:59.540539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:24:59.540817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:24:59.542525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:24:59.603304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:24:59.659627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:24:59.661057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ed913-562e-4d04-a671-a61855ac4ade service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:24:59.680232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 730/1475] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:01.813578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:01.814147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:01.814283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:01.815655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 746/1476] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:01.820710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957a21-d9ef-41e9-9200-8f6f0f736b82 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:01.821120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957a21-d9ef-41e9-9200-8f6f0f736b82 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:01.821260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957a21-d9ef-41e9-9200-8f6f0f736b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:01.885306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9c957a21-d9ef-41e9-9200-8f6f0f736b82 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:01.886484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9c957a21-d9ef-41e9-9200-8f6f0f736b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:01.982179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 731/1477] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:02.005343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:02.006913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:02.133348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2861c5-3435-49d2-94bc-6542f8412b3a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:02.183548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:02.228325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:02.228988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:02.229272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:02.229549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:02.230336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:02.232073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:25:02.232073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:25:02.232073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:25:02.233208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746ace-b08a-4c26-9062-07b273d41e6b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:02.252466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 747/1478] 10.222.0.93 () {56 vars in 1108 bytes} [Thu Jul 14 14:25:02 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:25:04.320045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:04.320580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:04.320580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:04.321944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 732/1479] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:04.327220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cfb66-8279-4c4b-ab37-0b7be54f65a2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:04.327629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cfb66-8279-4c4b-ab37-0b7be54f65a2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:04.327817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cfb66-8279-4c4b-ab37-0b7be54f65a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:04.392588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e11cfb66-8279-4c4b-ab37-0b7be54f65a2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:04.393828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e11cfb66-8279-4c4b-ab37-0b7be54f65a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:04.490488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 748/1480] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:04.534011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:04.535533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:04.545234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:04.680823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed75ae-2626-45c4-bf4f-345dd44dea80 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:04.769522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:04.802549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:04.803216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:04.803500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:04.803776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:04.805399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:04.897732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:04.899344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac4136-8838-468c-afd4-3de843c39fff service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:04.917403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 733/1481] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:07.135115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:07.135748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:07.135748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:07.137057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 749/1482] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:07.143091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d15e-607f-49b1-bfbe-70aa7ff1afef None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:07.143563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d15e-607f-49b1-bfbe-70aa7ff1afef None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:07.143670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d15e-607f-49b1-bfbe-70aa7ff1afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:07.208526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4f03d15e-607f-49b1-bfbe-70aa7ff1afef None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:07.209745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4f03d15e-607f-49b1-bfbe-70aa7ff1afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:07.257843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:07.307389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 734/1483] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:07.344671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:07.346288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:07.372610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:07.458205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d064f4-ec8f-4386-8035-52c8765cbe99 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:07.532501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:07.533145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:07.533427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:07.533702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:07.535393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:07.609901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:07.645418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:07.646988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c2f9-bf5d-4f23-80ad-49615d30ce1c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:07.666209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 750/1484] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:09.741130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:09.741650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:09.741754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:09.743079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 735/1485] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:09.748314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941cb43-7966-4b5d-9f07-25e9a8f86e76 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:09.748770 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941cb43-7966-4b5d-9f07-25e9a8f86e76 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:09.748871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941cb43-7966-4b5d-9f07-25e9a8f86e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:09.814379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5941cb43-7966-4b5d-9f07-25e9a8f86e76 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:09.815595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5941cb43-7966-4b5d-9f07-25e9a8f86e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:09.913317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 751/1486] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:09.951133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:09.952674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:10.054778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8195e7-1a50-4256-b5f1-746a90ac0f06 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:10.126566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:10.128444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:10.129107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:10.129389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:10.129664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:10.131340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:10.238664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:10.240142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e24a8-c557-4a6e-b9b1-376c4957ace2 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:10.258847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 736/1487] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:25:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:12.380277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:12.380825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:12.380825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:12.382192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 752/1488] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:12.387434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105015-6ff9-430b-a915-469f972f3e3b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:12.387866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105015-6ff9-430b-a915-469f972f3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:12.388004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105015-6ff9-430b-a915-469f972f3e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:12.451941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-14105015-6ff9-430b-a915-469f972f3e3b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:12.453142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-14105015-6ff9-430b-a915-469f972f3e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:12.507240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:12.550603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 737/1489] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:12.590591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:12.592126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:12.628983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:12.697169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee4df6-2b24-4e1a-9c69-2514e8fdef3b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:12.770616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:12.771268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:12.771556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:12.771835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:12.773551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:12.859422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:12.890357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:12.891842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f8a4-65b6-4dd1-96f7-9a2d9c3ab10b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:12.910696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 753/1490] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:15.443221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:15.443792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:15.443792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:15.445278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 738/1491] 10.222.0.93 () {54 vars in 926 bytes} [Thu Jul 14 14:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:25:15.450884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4cc0-1c42-4f5b-94c4-32fddba044a2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:15.451340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4cc0-1c42-4f5b-94c4-32fddba044a2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:15.451507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4cc0-1c42-4f5b-94c4-32fddba044a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:15.520106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f48a4cc0-1c42-4f5b-94c4-32fddba044a2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:25:15.522429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f48a4cc0-1c42-4f5b-94c4-32fddba044a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:15.631026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 754/1492] 10.222.0.93 () {58 vars in 1023 bytes} [Thu Jul 14 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:25:15.670211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:25:15.671747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:15.775158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c1311-2367-4fba-b147-300dd618e3a5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:25:15.847700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:25:15.848347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:25:15.848631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:25:15.848906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:25:15.850588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:25:15.860101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:15.960220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:25:15.961671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130a61f-501c-44af-861c-f62ac70aee32 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:25:15.979577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 739/1493] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:25:25.629215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:25:25.977348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=50 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:55.173430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:55.174065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:55.174065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:55.251092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:26:55.252325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:55.349650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 755/1494] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:26:55.358660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:55.359183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:55.359286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:55.431908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:26:55.433104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:55.528031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 740/1495] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:26:55.533237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:55.534544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:55.546736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f43b38-c424-46c6-a07b-097e9f52ebcf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:55.606151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:55.639942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:55.640884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:55.641476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:55.641795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:55.642535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:26:55.642852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:55.643881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:55.643881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:26:55.643881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:26:55.644773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:55.656841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 756/1496] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:26:55.665107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:55.674999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fdbe7-567d-4f19-acf2-962aa005ce2b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:55.737022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:55.770215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:55.770876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:55.771164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:55.771445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:55.772152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:55.773932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:55.773932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:26:55.773932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:26:55.775135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:55.959109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 741/1497] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:26:55 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:26:55.963408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:55.965012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:56.016945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d20452-1420-4aef-8ce1-6dcf92af7623 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:56.110729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:56.111404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:56.111690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:56.111968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:56.112669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:56.113674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:56.113674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:26:56.113674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:26:56.114915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:56.282559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 757/1498] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:26:55 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:26:56.286732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:56.288231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:56.360328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605dd6-94d0-465c-9f17-1c412aef78c6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:56.441415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:56.451682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:56.453387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:56.453387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:56.453387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:56.453758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:56.454597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:56.454597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:26:56.454597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:26:56.455472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:56.467949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 742/1499] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:26:56.472172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:56.473741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:56.486515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c555-0545-4f0d-a958-cb3a41af299c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:56.512609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:56.580780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:56.581395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:56.581667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:56.581974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:56.582697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:56.583738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:56.583738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:26:56.583738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:26:56.584641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3e4be-a80c-4529-b01a-13f18ec37e60 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:56.597012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 758/1500] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:26:56.601252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:56.602833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:56.638544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b77c-1be5-48c9-8240-164bbe2ad1b9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:56.731622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:56.733046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:56.733585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:56.733898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] PATH_INFO: `/v3/projects/57812d4683f542909c53f03a385846b5/users/2ad7e31090204a84b4210a8e44e0bb86/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:56.756929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57812d4683f542909c53f03a385846b5, user_id=2ad7e31090204a84b4210a8e44e0bb86, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:56.758065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:56.758065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:26:56.758065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:26:56.760011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa471e4-b542-45eb-a8ee-1b53c296a76a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:56.883539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 743/1501] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:26:56 2022] PUT /identity/v3/projects/57812d4683f542909c53f03a385846b5/users/2ad7e31090204a84b4210a8e44e0bb86/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:26:56.895357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98ae4-31b1-4340-b9a8-24ae77d05a4b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:56.895652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98ae4-31b1-4340-b9a8-24ae77d05a4b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:56.895934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98ae4-31b1-4340-b9a8-24ae77d05a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:56.951103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-40f98ae4-31b1-4340-b9a8-24ae77d05a4b None None] MFA Rules not processed for user `2ad7e31090204a84b4210a8e44e0bb86`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:26:56.952300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-40f98ae4-31b1-4340-b9a8-24ae77d05a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:57.030265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 759/1502] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:26:57.043424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:57.043745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:57.044032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:57.046019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 744/1503] 10.222.0.93 () {54 vars in 963 bytes} [Thu Jul 14 14:26:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:26:57.051052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5377-cace-43ed-9e72-3bddb5ff2940 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:57.051347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5377-cace-43ed-9e72-3bddb5ff2940 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:57.051630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5377-cace-43ed-9e72-3bddb5ff2940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:57.081359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:57.127321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2f3e5377-cace-43ed-9e72-3bddb5ff2940 None None] MFA Rules not processed for user `398d1954cb704f3898d3984f5c6102a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:26:57.128777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2f3e5377-cace-43ed-9e72-3bddb5ff2940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:57.209237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 760/1504] 10.222.0.93 () {58 vars in 1060 bytes} [Thu Jul 14 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:26:57.217863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:57.219461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:57.269186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:57.323451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7a80-6267-48ab-ac36-c6b059f25698 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:57.396743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:57.397432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:57.397667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:57.397992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:57.399646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:57.498250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:57.499854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:57.503518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:57.518490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 745/1505] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:26:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:26:57.560473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f973d03-db48-44a1-8419-bdeb16ea9416 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:57.560919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f973d03-db48-44a1-8419-bdeb16ea9416 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:57.561024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f973d03-db48-44a1-8419-bdeb16ea9416 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:57.562337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 761/1506] 10.222.0.93 () {52 vars in 829 bytes} [Thu Jul 14 14:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:26:57.565750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:57.566987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:57.667108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e408f-6044-40f5-b584-fd4d71a032ed service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:57.759786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:57.760568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:57.760851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:57.761125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] PATH_INFO: `/v3/users/2ad7e31090204a84b4210a8e44e0bb86` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:57.773823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad7e31090204a84b4210a8e44e0bb86)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:57.774855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:57.774855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:26:57.774855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:26:57.776065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:57.785705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:58.047497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad7e31090204a84b4210a8e44e0bb86'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:58.066670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad7e31090204a84b4210a8e44e0bb86'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:58.150760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad7e31090204a84b4210a8e44e0bb86'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:58.151294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad7e31090204a84b4210a8e44e0bb86'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:58.151718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad7e31090204a84b4210a8e44e0bb86'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:58.170053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 746/1507] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:26:57 2022] DELETE /identity/v3/users/2ad7e31090204a84b4210a8e44e0bb86 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:26:58.174467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:58.174757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:58.175098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:58.229365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:26:58.230656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:58.308378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:58.320659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 762/1508] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:26:58.332175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:58.333393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:58.432254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d37-20f6-46cd-bf2b-ea9fe4a0c69f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:58.486549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:26:58.504717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:58.505364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:58.505647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:58.505965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:58.507602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:58.617948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:58.619341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceef5be-6fd3-43a7-a49a-aaa256b6eeb1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:58.637125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 747/1509] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:26:59.333388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:26:59.333388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:26:59.343883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ef61c-f5cc-4a75-b098-0e5d3559aeab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:26:59.446399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:26:59.447259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:26:59.447543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:26:59.447820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] PATH_INFO: `/v3/projects/57812d4683f542909c53f03a385846b5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:26:59.458116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57812d4683f542909c53f03a385846b5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:26:59.459213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:26:59.459213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:26:59.459213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:26:59.460503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:26:59.582644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57812d4683f542909c53f03a385846b5', 'request_id': 'req-df0d3ae7-299c-48e7-8dd8-24ebe2850428'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:59.595359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57812d4683f542909c53f03a385846b5', 'request_id': 'req-df0d3ae7-299c-48e7-8dd8-24ebe2850428'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:59.697713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57812d4683f542909c53f03a385846b5', 'request_id': 'req-df0d3ae7-299c-48e7-8dd8-24ebe2850428'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:59.720820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] The token cache is being invalidate because project 57812d4683f542909c53f03a385846b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:26:59.721106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:59.721627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-df0d3ae7-299c-48e7-8dd8-24ebe2850428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:26:59.723018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 763/1510] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:26:59 2022] DELETE /identity/v3/projects/57812d4683f542909c53f03a385846b5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:27:08.634677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=62 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:27:09.718810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:28:37.913014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:37.913652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:37.913652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:37.915088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:37.920530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cf77b-8da0-4cfb-9406-fb1c64e09583 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:37.920968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cf77b-8da0-4cfb-9406-fb1c64e09583 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:37.921075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cf77b-8da0-4cfb-9406-fb1c64e09583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:38.010771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fe4cf77b-8da0-4cfb-9406-fb1c64e09583 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:38.012821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fe4cf77b-8da0-4cfb-9406-fb1c64e09583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:38.134836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:38.304588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:38.307024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:38.438953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9b434-66a1-4177-a278-dab3b3a46643 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:38.444617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:38.445267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:38.445558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:38.445881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:38.454038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:38.455642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01347c-c593-4d7c-a21b-301083769e85 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:38.465010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 749/1513] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:39.331071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb06b8-2c3a-42ae-b06e-be455bf4acdf None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:39.331649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb06b8-2c3a-42ae-b06e-be455bf4acdf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:39.331649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb06b8-2c3a-42ae-b06e-be455bf4acdf None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:39.333023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:39.338545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a905-41f4-49af-9c85-367927425a6d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:39.339011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a905-41f4-49af-9c85-367927425a6d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:39.339119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a905-41f4-49af-9c85-367927425a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:39.356827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8435a905-41f4-49af-9c85-367927425a6d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:39.358841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8435a905-41f4-49af-9c85-367927425a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:39.367348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:39.524799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb06b8-2c3a-42ae-b06e-be455bf4acdf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:39.544091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb06b8-2c3a-42ae-b06e-be455bf4acdf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:39.552324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:39.553283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:39.553687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:39.554136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:39.566943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:39.571139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac9e2e-cd4f-4e5e-b1fa-f172e25c0ffc service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:39.581724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 766/1516] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:39.993672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e157f-2fe9-4c6a-92b0-f9aa2e70d952 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:39.993999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e157f-2fe9-4c6a-92b0-f9aa2e70d952 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:39.994283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e157f-2fe9-4c6a-92b0-f9aa2e70d952 None None] PATH_INFO: `/v3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:39.995596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 751/1517] 10.222.0.91 () {54 vars in 925 bytes} [Thu Jul 14 14:28:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:28:40.001954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3fd33-7357-4b30-b68b-14087e2cd467 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:40.007107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3fd33-7357-4b30-b68b-14087e2cd467 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:40.007107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3fd33-7357-4b30-b68b-14087e2cd467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:40.061391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54a3fd33-7357-4b30-b68b-14087e2cd467 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:40.062642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54a3fd33-7357-4b30-b68b-14087e2cd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:40.080359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:28:40.135740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 767/1518] 10.222.0.91 () {58 vars in 1015 bytes} [Thu Jul 14 14:28:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:44.442750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c5be-ce45-4bc0-8219-58c3df5f2cac None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:44.443304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c5be-ce45-4bc0-8219-58c3df5f2cac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:44.443304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c5be-ce45-4bc0-8219-58c3df5f2cac None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:44.444751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 752/1519] 10.222.0.93 () {54 vars in 960 bytes} [Thu Jul 14 14:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:44.449743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabde7a-cd10-4182-896c-0a627702f9e6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:44.450208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabde7a-cd10-4182-896c-0a627702f9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:44.450316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabde7a-cd10-4182-896c-0a627702f9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:44.468424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0aabde7a-cd10-4182-896c-0a627702f9e6 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:44.469615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0aabde7a-cd10-4182-896c-0a627702f9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:44.478003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 768/1520] 10.222.0.93 () {58 vars in 1057 bytes} [Thu Jul 14 14:28:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:44.486040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18c5be-ce45-4bc0-8219-58c3df5f2cac None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:44.500176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18c5be-ce45-4bc0-8219-58c3df5f2cac None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:44.505252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29609c26-5429-4372-88f8-364901484695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:44.505918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609c26-5429-4372-88f8-364901484695 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:44.506209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609c26-5429-4372-88f8-364901484695 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:44.506488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609c26-5429-4372-88f8-364901484695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:44.514817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29609c26-5429-4372-88f8-364901484695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:44.516396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29609c26-5429-4372-88f8-364901484695 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:44.526860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 753/1521] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:50.800427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa256c-49ee-4506-bafb-be15addcbb73 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:50.801020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa256c-49ee-4506-bafb-be15addcbb73 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:50.801020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa256c-49ee-4506-bafb-be15addcbb73 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:50.802371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:50.807089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3544-746d-4f5d-a80b-0c361dd1f213 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:50.807512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3544-746d-4f5d-a80b-0c361dd1f213 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:50.807662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3544-746d-4f5d-a80b-0c361dd1f213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:50.825331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1d0e3544-746d-4f5d-a80b-0c361dd1f213 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:50.826570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1d0e3544-746d-4f5d-a80b-0c361dd1f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:50.834400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:50.989498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa256c-49ee-4506-bafb-be15addcbb73 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:51.004574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa256c-49ee-4506-bafb-be15addcbb73 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:51.010261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:51.010901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:51.011196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:51.011477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:51.020294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:51.021952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2de41-c5c4-4bb1-bdd4-3b5e1e488ddc service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:51.031613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 770/1524] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:51.859813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021c6e4-0a34-4d96-959f-a943a05f6d2a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:51.860398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021c6e4-0a34-4d96-959f-a943a05f6d2a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:51.860398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021c6e4-0a34-4d96-959f-a943a05f6d2a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:51.861745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 14:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:51.866716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48eb30-3ced-48ea-9c1e-0425c469c399 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:51.867228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48eb30-3ced-48ea-9c1e-0425c469c399 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:51.867326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48eb30-3ced-48ea-9c1e-0425c469c399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:51.885636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ab48eb30-3ced-48ea-9c1e-0425c469c399 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:51.886870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab48eb30-3ced-48ea-9c1e-0425c469c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:51.894812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 14:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:28:52.057440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021c6e4-0a34-4d96-959f-a943a05f6d2a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:52.072432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021c6e4-0a34-4d96-959f-a943a05f6d2a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:52.077836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:52.078476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:52.078770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:52.079046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:52.087576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:52.089167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46ddf2-f41f-405c-8981-d0905eaff8d0 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:52.098702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 756/1527] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:56.094697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:56.095236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:56.095362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:56.096652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:56.101987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4e2d-542f-48ef-bb3c-1d2e1f26da99 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:56.102419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4e2d-542f-48ef-bb3c-1d2e1f26da99 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:56.102558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4e2d-542f-48ef-bb3c-1d2e1f26da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:56.120709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-31ba4e2d-542f-48ef-bb3c-1d2e1f26da99 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:56.121952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-31ba4e2d-542f-48ef-bb3c-1d2e1f26da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:56.130264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:56.202934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:56.204636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:56.284407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37794c-2b11-4a33-8742-ef48702ee7e6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:56.288209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:56.288853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:56.289137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:56.289414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:56.298956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:56.300503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c24ee4-0f9b-441d-8fda-f019e79b375c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:56.309832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 773/1530] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:28:58.320753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0955a-a097-40d1-9683-338c75cf6aa5 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:58.321289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0955a-a097-40d1-9683-338c75cf6aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:58.321289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0955a-a097-40d1-9683-338c75cf6aa5 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:58.322615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:28:58.328243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a83733-d7bc-4613-963f-3bdaba3ec5ae None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:58.328663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a83733-d7bc-4613-963f-3bdaba3ec5ae None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:58.328807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a83733-d7bc-4613-963f-3bdaba3ec5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:58.346028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-99a83733-d7bc-4613-963f-3bdaba3ec5ae None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:28:58.347222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-99a83733-d7bc-4613-963f-3bdaba3ec5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:28:58.355007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:28:58.392052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0955a-a097-40d1-9683-338c75cf6aa5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:28:58.402633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0955a-a097-40d1-9683-338c75cf6aa5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:28:58.406078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:28:58.406704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:28:58.406986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:28:58.407264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:28:58.416024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:28:58.417444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208d863-df84-490c-b736-183bc47e5a95 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:28:58.426658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 759/1533] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:29:01.944691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:01.945278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:01.945278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:01.946702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:29:01.952557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443010-3bee-4cc6-8994-3dfcef11de69 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:01.952986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443010-3bee-4cc6-8994-3dfcef11de69 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:01.953111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443010-3bee-4cc6-8994-3dfcef11de69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:01.959821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=81 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:29:01.971568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5c443010-3bee-4cc6-8994-3dfcef11de69 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:29:01.972783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c443010-3bee-4cc6-8994-3dfcef11de69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:29:01.980682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:29:02.290800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:29:02.292489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:29:02.359748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:29:02.372809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b8a5c-277f-4d4f-98d9-9882258cea0d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:29:02.376643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:29:02.377057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:02.377347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:02.377628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:02.386745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:29:02.388697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d3024-a713-4abe-95ed-7be41a456a20 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:29:02.398196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 776/1536] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:29:04.559306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:04.560670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:04.560830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:04.562363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 14:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:29:04.567664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bc9b-6032-43d8-967d-8ac96d997c06 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:04.568112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bc9b-6032-43d8-967d-8ac96d997c06 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:04.568226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bc9b-6032-43d8-967d-8ac96d997c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:04.586372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-34a8bc9b-6032-43d8-967d-8ac96d997c06 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:29:04.587651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34a8bc9b-6032-43d8-967d-8ac96d997c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:29:04.596053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 14:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:29:04.897750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:29:04.899486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:29:04.913368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c06af5-ade5-4087-983d-09a59d93abde None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:29:04.916928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:29:04.917568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:29:04.917888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:29:04.918169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:29:04.927201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:29:04.928631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467452a-6c3a-471b-a8f6-63aa14fd494f service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:29:04.938572 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 762/1539] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:29:14.581461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=11 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:29:14.933958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=8 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:34.121160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.121160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.121160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.151081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:34.153209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:34.175419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 778/1540] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:34.185416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.186014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.186014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.224951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:34.226190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:34.251879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 763/1541] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:34.256844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.283074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a55025-46a9-4d45-8caa-e66574775cc4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.307182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.308721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.309213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.309695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.310674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:34.311187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.314933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66084166-c935-4f03-aadf-c400abc577a7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.338112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 779/1542] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:34.346440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.367489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3359df-06de-4c36-82d5-e5de1ef26b06 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.390616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.391328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.391802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.392091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.392845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.394902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.511479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1da80-bb07-447d-a6c7-638ffa0656ad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.511479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1da80-bb07-447d-a6c7-638ffa0656ad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.511479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1da80-bb07-447d-a6c7-638ffa0656ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.540011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 764/1543] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:34.546025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.552420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7be1da80-bb07-447d-a6c7-638ffa0656ad None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:34.553204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7be1da80-bb07-447d-a6c7-638ffa0656ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:34.663369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d12c-c923-4936-a1f6-b75ee8da4284 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.693835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 780/1544] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:34.698522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.699221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.699513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.699796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.700505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.701636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:34.701636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:30:34.701636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:30:34.703004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.703482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.703932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.704349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.743069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:34.749935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:34.772052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 781/1545] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:34.780617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.803919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bd0b9-1a4a-4b04-a048-054b755e1482 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.823445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.824509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.826825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.827473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.828497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:34.829021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.830930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.839328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 765/1546] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/users => generated 287 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:34.847147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.855683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 782/1547] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:34.867054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:34.868386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4797c4-ce11-498d-9c75-1cc7f8992730 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.890898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.891774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.892069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.892352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.893102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.894839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939819-3b59-45f7-b02e-382a22146fbc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:34.903089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2c94e-172c-4f88-8e9a-2d6f29576b4d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.917152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:34.917152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.917152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.917152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.917630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:34.918687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:34.918687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:30:34.918687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:30:34.920210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f5500-b3b7-4561-81f8-1ff4b3b2ddac admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:34.929816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 766/1548] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:34.929816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:34.929816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:34.929816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:34.973826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:34.976855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.001908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 767/1549] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.003259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.015459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace733a-1cd6-4544-872a-579e12446f81 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.028286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.029407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.029986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.030480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.031445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.038524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad1e5f-5a96-4e32-ba53-17f9710df36a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.051550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 768/1550] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:35.053336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf951-edef-4845-931a-7f953cbf912d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.053893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf951-edef-4845-931a-7f953cbf912d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.054414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf951-edef-4845-931a-7f953cbf912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.086204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-35caf951-edef-4845-931a-7f953cbf912d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.090660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35caf951-edef-4845-931a-7f953cbf912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.105750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 783/1551] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:34 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:35.107828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06f6ec-720b-45e9-ab07-78799db7d3a9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.108487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06f6ec-720b-45e9-ab07-78799db7d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.109096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06f6ec-720b-45e9-ab07-78799db7d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.119582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 769/1552] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.121627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.122038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.122455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.149264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ec06f6ec-720b-45e9-ab07-78799db7d3a9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.153715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.155077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.163431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ec06f6ec-720b-45e9-ab07-78799db7d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.246968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 770/1553] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.247821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.254462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 784/1554] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.261981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.261981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.261981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.262276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8bd50-7ea0-4a9b-8f9f-2989de68365e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.265612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.266924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.267216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.267494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] PATH_INFO: `/v3/projects/00aaa9fa0a0949188f8a80bf052d8105/users/097d2584618a4292ac559e83a321669b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.277651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.278988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.287398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 785/1555] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.288281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.289054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00aaa9fa0a0949188f8a80bf052d8105, user_id=097d2584618a4292ac559e83a321669b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.292039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.294944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:35.299949 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb796bef-3cf9-400b-b9ac-ff98c9e17f90 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.304787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.308868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.308868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.308868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.308868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.309349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa35aa-67e2-4539-8528-32843d27c438 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.439964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 771/1556] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:35 2022] PUT /identity/v3/projects/00aaa9fa0a0949188f8a80bf052d8105/users/097d2584618a4292ac559e83a321669b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:35.440316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 786/1557] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:35.440847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.441599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf24-5d1e-4197-9288-fc3f6e904af4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.441827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf24-5d1e-4197-9288-fc3f6e904af4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.442114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf24-5d1e-4197-9288-fc3f6e904af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.465175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4eddbf24-5d1e-4197-9288-fc3f6e904af4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.466190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4eddbf24-5d1e-4197-9288-fc3f6e904af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.487126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:35.513560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed199fd8-59b5-4e85-9afb-43a296ffb424 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.522258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.523284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.523588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.523869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.524608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.524920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.525979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:35.525979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:30:35.525979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:30:35.526864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a4eaf-2899-4586-ada8-fd2e39c7d551 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.539491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 772/1558] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:35.542613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b610e-8a3e-4a6e-a70f-167fa47d36f6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.542924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b610e-8a3e-4a6e-a70f-167fa47d36f6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.543188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b610e-8a3e-4a6e-a70f-167fa47d36f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.555824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 787/1559] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.562236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8af21-c2bf-427e-83de-d287d34c1b15 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.562236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8af21-c2bf-427e-83de-d287d34c1b15 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.562236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8af21-c2bf-427e-83de-d287d34c1b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.577021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3c5b610e-8a3e-4a6e-a70f-167fa47d36f6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.578271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3c5b610e-8a3e-4a6e-a70f-167fa47d36f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.586893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 773/1560] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.588259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5691-1401-4008-804e-5e9c54e05b23 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.588521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5691-1401-4008-804e-5e9c54e05b23 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.589151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5691-1401-4008-804e-5e9c54e05b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.592458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-62f8af21-c2bf-427e-83de-d287d34c1b15 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.593703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-62f8af21-c2bf-427e-83de-d287d34c1b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.606054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 788/1561] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.607896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.608195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.608486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.613980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-863f5691-1401-4008-804e-5e9c54e05b23 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.615338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-863f5691-1401-4008-804e-5e9c54e05b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.625647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 774/1562] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.627008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0783-2810-41a1-9bea-77f1b7678e38 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.627298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0783-2810-41a1-9bea-77f1b7678e38 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.627581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0783-2810-41a1-9bea-77f1b7678e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.629019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.630404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.640603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 789/1563] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.641424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.652746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40381f08-6127-4267-b172-e1826c7e8ef5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.656378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3f7f0783-2810-41a1-9bea-77f1b7678e38 None None] MFA Rules not processed for user `097d2584618a4292ac559e83a321669b`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.657757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f7f0783-2810-41a1-9bea-77f1b7678e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.659056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.660136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.660261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.660517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.661252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.662917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.674511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 790/1564] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:35.674511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.686756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a909c-d630-4e84-aeac-c7d01bd0db33 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.691955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.692620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.692910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.693198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.694164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.696095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.736057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 775/1565] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.737357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae4e2a-695c-407c-a9e7-6bb147999a04 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.737618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae4e2a-695c-407c-a9e7-6bb147999a04 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.737968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae4e2a-695c-407c-a9e7-6bb147999a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.756100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c6ae4e2a-695c-407c-a9e7-6bb147999a04 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.757309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c6ae4e2a-695c-407c-a9e7-6bb147999a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.765815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 776/1566] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.771277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.771762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.771967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.792467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:35.795986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:35.809266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 777/1567] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:35.809266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.814114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64ee39-1819-42b1-a623-550f4ad5b47a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.817804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 791/1568] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/projects => generated 391 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:35.818467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.819331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.820001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.820440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.820794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.821454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.821784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.823624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.837493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 778/1569] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:35.838244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.855039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a7e3-4bd4-46ec-9319-bbc954efa9b4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.894045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fb5f-bda6-4a83-a355-4ef587f4c3a0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.894291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:35.903507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.905524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.905524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.905524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.906036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.906313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.908447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.924901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.926959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.926959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.927180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 792/1570] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:35.927307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.927307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.927307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.927775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.928944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a321b2-00c8-42a0-9c9d-c4b97b1f7186 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.939551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662f0-49d1-4ebd-baa4-e10d3a475daf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.944647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 779/1571] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:35.944799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.945596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.945962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.946098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497e94-2b7a-4ea0-88aa-c3aabb928a34 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.946330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.946518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497e94-2b7a-4ea0-88aa-c3aabb928a34 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.946688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497e94-2b7a-4ea0-88aa-c3aabb928a34 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.947002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.947851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 780/1572] 10.222.0.93 () {54 vars in 961 bytes} [Thu Jul 14 14:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:35.948583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3497e94-2b7a-4ea0-88aa-c3aabb928a34 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.953140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.964398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3497e94-2b7a-4ea0-88aa-c3aabb928a34 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.965876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 793/1573] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:35.978696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.978821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.978821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.978821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.978821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.979149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.979149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.979149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.979947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765f2a-0c0b-4719-994d-0727e8d3106e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:35.984112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:35.984810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:35.985095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:35.985406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:35.986312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:35.986590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:35.987216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 781/1574] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:35.987940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:35.995441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:35.998162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf7acb-7255-4d2b-990f-ffc637461183 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.003744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.004421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.004729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.005011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.005715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.007996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.009454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 794/1575] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:36.010386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.025143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bef9-2da1-4c5f-802e-f4c5c896b5ff admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.030741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.031396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.031692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.031967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.032722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.034742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.160312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 782/1576] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:35 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.161282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.172251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a1608-7cc2-448e-b5a5-48836762cd7f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.177178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.177882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.178184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.178458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.179194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.181159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3b45a-31ee-462c-b69e-b9343f453201 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.211988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 795/1577] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.216431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.234832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:36.278563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29002e18-27d3-443b-8adf-c616ca10a209 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.284134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.284818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.285107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.286792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.287480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.289448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.352184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 783/1578] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.353336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.353647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.353952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.395383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:36.396634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:36.420715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 796/1579] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.421623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.471791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 784/1580] 10.222.0.93 () {58 vars in 1058 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:36.472406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.497498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838938d-f823-48aa-aae9-05695614c1ba admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.503379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.504386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.504678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.505142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] PATH_INFO: `/v3/projects/a344d7e2cadb46129ede551661428bc7/users/b209aeb321f64346a6d85d6ebebdd637/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.522644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a344d7e2cadb46129ede551661428bc7, user_id=b209aeb321f64346a6d85d6ebebdd637, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.523703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:36.523703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:30:36.523703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:30:36.525705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.534865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:36.539256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d0c4-5ea4-445d-9878-e24c584d0479 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.545498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.545498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.545498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.545818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.546274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.548175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.668552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 797/1581] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:36 2022] PUT /identity/v3/projects/a344d7e2cadb46129ede551661428bc7/users/b209aeb321f64346a6d85d6ebebdd637/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:36.670051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.700725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 785/1582] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.700725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.744234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b8e15-b471-434e-9141-a0c09fad2edd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.777090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687d3cf-fc04-498d-9393-91e0ae608946 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.781761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:36.785052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.786033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.786475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.787071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.788028 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.789806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.800592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 786/1583] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:36.806517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.808117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.808117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.808117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.808117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.809003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.811024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.813803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936e0e3-7e30-4147-9e0e-4e955122462c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.818719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.819475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.820975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.821399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.822222 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.824740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:36.895929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 798/1584] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.896953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.929383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 787/1585] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:36.937198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:36.940353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7744a6-680b-4f7d-998d-ec2b84ad133b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.967112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017616a4-0c95-4bf3-9cbf-3d4b13347bef admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:36.971918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:36.972600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:36.972895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:36.973211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:36.973951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:36.976484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.005225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.005950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.006298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.006584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.007336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.009345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80c059-a596-4d64-9e7b-53d45fb60b50 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.048009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 799/1586] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:37.049301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.129187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413e15d-4f02-417b-ad3c-c1962f529460 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.137950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.138705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.139079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.139428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.140873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 788/1587] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:36 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:37.142262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.142554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.142842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.178092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] MFA Rules not processed for user `b209aeb321f64346a6d85d6ebebdd637`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:37.179499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:37.206245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:37.219183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.220917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.227580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:37.230917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 800/1588] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:37.231965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.246893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d68f-40ca-4b4a-a2d2-b531ca7baa8b service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.255875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.256760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 789/1589] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:37.256897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.257260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.257621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.257840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.258680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.261673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.269123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71ffc6-9b43-4465-849e-b4adac070f68 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.276375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.280298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.280298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.280298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.280298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.284165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.300272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 790/1590] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.300272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.312703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9d72-9162-4c4e-a43a-f7cbf084b6d4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.318125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.318953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.319238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.319517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.320211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.322346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.371334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 801/1591] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:37 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:37.372382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.383602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b4e4-c361-4a62-b5bb-f0aab8d7814a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.390221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.390946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.391242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.391525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.392292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.394297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.407075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 802/1592] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.408167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.419163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d035-4c61-43d8-b66e-de232f43d5e6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.424135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.424866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.425161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.425434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.426397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.428015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.440999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 803/1593] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.445178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.452390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2975d-57f3-47e3-824b-2ff15293c956 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.457988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.458658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.458942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.459218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.460123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.461876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef53089-f984-478a-b834-a8048d62d9b3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.473577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 804/1594] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.474870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf4ed-7ace-4da9-9275-a82b10f6d92b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.475147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf4ed-7ace-4da9-9275-a82b10f6d92b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.475442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf4ed-7ace-4da9-9275-a82b10f6d92b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.482008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 805/1595] 10.222.0.93 () {54 vars in 961 bytes} [Thu Jul 14 14:30:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:37.482901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebf4ed-7ace-4da9-9275-a82b10f6d92b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.502623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebf4ed-7ace-4da9-9275-a82b10f6d92b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.505818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 791/1596] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:37 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:37.505818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.508259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.509628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.510214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.510696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] PATH_INFO: `/v3/projects/e2578b0e75cb455c93cf83dfe54842ff/users/be6cba32a31a425e964d7222c3d6b9fc/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.527979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477bb3d-2da6-49cd-a961-49c94c432a8b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.534109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.534737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.535118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.535529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.539981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.544017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.546557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2578b0e75cb455c93cf83dfe54842ff, user_id=be6cba32a31a425e964d7222c3d6b9fc, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.549230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852a91-5825-4b3a-bbbb-1dfd7a577497 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.564708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 792/1597] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.565561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.578108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc051-b8d2-4a35-8fb5-5bb8afc1bc96 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.583456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.584264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.584793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.585144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.586020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.587745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.598727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 793/1598] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.599611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.611866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931282b-4e43-424a-b57f-4d36b98f89ce admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.616513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.617249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.617605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.617986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.618713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.621286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.632391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 794/1599] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.633285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.643090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70da51-1252-4980-8a10-323a0f7d8b60 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.647876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.648556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.648841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.649118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.649915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.651540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea4977-3e64-495e-9c95-f0d4252532f3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.662797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 795/1600] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.664032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.664329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.664606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.684117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:37.685361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:37.694988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 796/1601] 10.222.0.93 () {58 vars in 1058 bytes} [Thu Jul 14 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:37.695785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.707967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37607e-0b94-48f0-92fb-317679c3df85 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.712520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.713182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.713473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.713861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.714605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.716317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.730689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 797/1602] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.730689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.743388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6346-2596-4de5-b95b-4cc00e0e8605 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.744406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 806/1603] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:37 2022] PUT /identity/v3/projects/e2578b0e75cb455c93cf83dfe54842ff/users/be6cba32a31a425e964d7222c3d6b9fc/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:37.746139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ef61-95b8-49e4-b87a-0765e6691a9d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.746429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ef61-95b8-49e4-b87a-0765e6691a9d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.746707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ef61-95b8-49e4-b87a-0765e6691a9d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.748440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 807/1604] 10.222.0.93 () {54 vars in 912 bytes} [Thu Jul 14 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:37.748924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ef61-95b8-49e4-b87a-0765e6691a9d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.818470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.819492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.819849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.820366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.821188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.823249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.830704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ef61-95b8-49e4-b87a-0765e6691a9d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.837871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 798/1605] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:37.837871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:37.838142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.838768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.839064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.839349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] PATH_INFO: `/v3/projects/9f707e1171364031b016d16a10efda78/users/bc26c3a3096b4b318d18d4c919f7a60a/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.848770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aedee3-8c70-4f35-b8bd-c2e80fb012a9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:37.857658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f707e1171364031b016d16a10efda78, user_id=bc26c3a3096b4b318d18d4c919f7a60a, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.859505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:37.861100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:37.861386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:37.861704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:37.862159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] PATH_INFO: `/v3/projects/beb09664df9647519b7a1ed5a30b3858/users/4b08d1bedbc946868c9f9bc0b3808591/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:37.879187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb09664df9647519b7a1ed5a30b3858, user_id=4b08d1bedbc946868c9f9bc0b3808591, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:37.881935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.033730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 808/1606] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:37 2022] PUT /identity/v3/projects/9f707e1171364031b016d16a10efda78/users/bc26c3a3096b4b318d18d4c919f7a60a/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:38.034596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.096617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:38.111465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973be9be-f0bc-4030-b45b-b52c5d669ab8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.125259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 799/1607] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:37 2022] PUT /identity/v3/projects/beb09664df9647519b7a1ed5a30b3858/users/4b08d1bedbc946868c9f9bc0b3808591/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:38.125259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.157463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:38.183626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.184532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.184812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.185092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] PATH_INFO: `/v3/projects/fc023ab14cbc4b488105759e7aeb7e40/users/b83232101c954b88bc2dfb4c040c60b5/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.203242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc023ab14cbc4b488105759e7aeb7e40, user_id=b83232101c954b88bc2dfb4c040c60b5, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.206401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.208151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0f42-e6be-4653-8440-5f825c1741d1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.215409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.216122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.216751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.216751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.281324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.282963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27583a5-b601-4347-bd68-eb9735aad1ba service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.288070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 809/1608] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:38 2022] PUT /identity/v3/projects/fc023ab14cbc4b488105759e7aeb7e40/users/b83232101c954b88bc2dfb4c040c60b5/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:38.288853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.295116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 800/1609] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:38.297102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e63e4-491e-483c-8413-c64fd4154b8c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.305615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e63e4-491e-483c-8413-c64fd4154b8c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.305615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e63e4-491e-483c-8413-c64fd4154b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.319800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5e4e63e4-491e-483c-8413-c64fd4154b8c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:38.321130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5e4e63e4-491e-483c-8413-c64fd4154b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:38.394662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79b10-dc90-43cd-8056-9f5108c26b17 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.399697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.401305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.401601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.402401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.403173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.404766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.412185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 801/1610] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:38.414699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656179c-fb38-4c27-adf7-162423bb9e34 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.422322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 810/1611] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:38.422322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.422546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656179c-fb38-4c27-adf7-162423bb9e34 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.422546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656179c-fb38-4c27-adf7-162423bb9e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.438922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f391ca7-9eb6-426f-b4b0-4a7ed331ab4c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.443490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.444384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.444671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.445421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] PATH_INFO: `/v3/projects/b366acdadbf840488244d6c6d4e3b2d6/users/c1e7f6385e08458f9026a319e07bd8e1/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.451017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2656179c-fb38-4c27-adf7-162423bb9e34 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:38.452238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2656179c-fb38-4c27-adf7-162423bb9e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:38.460831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b366acdadbf840488244d6c6d4e3b2d6, user_id=c1e7f6385e08458f9026a319e07bd8e1, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.463887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f73b9-b6f1-45c4-9cd4-25dda53a2958 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.509307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:38.525660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 802/1612] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:38.527224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02947a1-6be7-4a46-8139-953570825a5c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.527647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02947a1-6be7-4a46-8139-953570825a5c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.527938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02947a1-6be7-4a46-8139-953570825a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.552073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 811/1613] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:38 2022] PUT /identity/v3/projects/b366acdadbf840488244d6c6d4e3b2d6/users/c1e7f6385e08458f9026a319e07bd8e1/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:38.553540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.555038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.555444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.574160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c02947a1-6be7-4a46-8139-953570825a5c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:38.575423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c02947a1-6be7-4a46-8139-953570825a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:38.594251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] MFA Rules not processed for user `4b08d1bedbc946868c9f9bc0b3808591`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:38.595481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:38.652821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:38.668516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 803/1614] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:38.670402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.670860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.671343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.683633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 812/1615] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:38.686764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.701487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6b23f-6008-4c08-a60f-023a3f9170d7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.705107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] MFA Rules not processed for user `b83232101c954b88bc2dfb4c040c60b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:38.707059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:38.709208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.710397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.710792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.711196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] PATH_INFO: `/v3/projects/8725a19566454345b77b639cfe170008/users/e2ac3e31fee448d7ae7a6d885f545636/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.732938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8725a19566454345b77b639cfe170008, user_id=e2ac3e31fee448d7ae7a6d885f545636, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.736092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e8816-0789-4078-8401-468da1bd69cc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:38.790788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 804/1616] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:38.792073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.850797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 813/1617] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:38 2022] PUT /identity/v3/projects/8725a19566454345b77b639cfe170008/users/e2ac3e31fee448d7ae7a6d885f545636/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:38.851347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe853-78c5-4ecb-85df-69d330353e48 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.852218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae882-d6f0-475c-b06d-450766f861a2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.852534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae882-d6f0-475c-b06d-450766f861a2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.852839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae882-d6f0-475c-b06d-450766f861a2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.854135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 814/1618] 10.222.0.93 () {54 vars in 912 bytes} [Thu Jul 14 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:38.856017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ae882-d6f0-475c-b06d-450766f861a2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:38.895838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:38.921820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.922798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.923136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.923522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.925215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ae882-d6f0-475c-b06d-450766f861a2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:38.929359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:38.930140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:38.930626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:38.931726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:38.995462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:38.997270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a7f0a-5156-4061-8b1a-494e01c7ad3c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.007947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 805/1619] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.008224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.009576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2a97d-f9d6-4ca7-869f-be0a719d6e9d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.010013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.010141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2a97d-f9d6-4ca7-869f-be0a719d6e9d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.010492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2a97d-f9d6-4ca7-869f-be0a719d6e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.024673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 815/1620] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.025626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.037524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dbe81-e18b-44c5-9c42-873fbc9c0cc8 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.042882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.043761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.044055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.044347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.049805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-11e2a97d-f9d6-4ca7-869f-be0a719d6e9d None None] MFA Rules not processed for user `c1e7f6385e08458f9026a319e07bd8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:39.055468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-11e2a97d-f9d6-4ca7-869f-be0a719d6e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:39.063419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.065047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.085088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 816/1621] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.086858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.101509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707b42-9d35-43cd-aa03-1c10e3df6c5f service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.108930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.184563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.184563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.184563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.184563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.184563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.185574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 806/1622] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:39.185574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea45f8-87b3-40a6-b075-174433f53c3a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.185574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea45f8-87b3-40a6-b075-174433f53c3a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.185574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea45f8-87b3-40a6-b075-174433f53c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.209275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1dea45f8-87b3-40a6-b075-174433f53c3a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:39.209275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1dea45f8-87b3-40a6-b075-174433f53c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:39.227964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 807/1623] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:39.229537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.230573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.231035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.240237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 817/1624] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:39.241104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.255016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:39.256495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:39.327781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:39.344800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 808/1625] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:39.351347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.366919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:39.376156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dceaa-bc3c-438a-8505-7e2ecdb453a4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.380462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5da5c-c1b6-46fe-832d-7880c166eee6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.386825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.387692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.388052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.388458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.389273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.390523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.391905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.392027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.392380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.393047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.454622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.456186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.466135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 809/1626] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.474776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.476280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:39.522238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 818/1627] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:39.522238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.596179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a122bf-b70e-4c9b-bb9d-86c749405346 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.625644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f923a-14eb-4de9-a878-09065b08e379 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.645082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.650417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.656724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.656724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.659764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.671010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.696709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.697502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.697807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.698133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.710098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.711487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.723962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 810/1628] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.724852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.794395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff328ea-5991-4e30-b5dc-3689937a10ca service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.803628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.804511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.805242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.805685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.820165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:39.826516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 819/1629] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:39.826516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.853556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1fb-5dd4-4c49-a8ce-bb7f871f1078 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.870382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.873752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.874976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.876075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.878348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.883627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.905617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.905617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.920843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 811/1630] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:39.922250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:39.946373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68a6b-f36a-4150-85f4-bfa89233f034 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:39.963328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:39.964855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:39.964855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:39.965087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:39.965801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:39.967630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:39.990386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 812/1631] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:39.992883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.010933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e3e3-bdfd-407f-906e-ea3affd84b3b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.023581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.025253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.025977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.027005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] PATH_INFO: `/v3/projects/3a6664c5660f406ca1d0934b1cd54159/users/b3a0c9d02e5541faa993610cc4f6b900/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.046056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6664c5660f406ca1d0934b1cd54159, user_id=b3a0c9d02e5541faa993610cc4f6b900, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.048842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.084734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 820/1632] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:39 2022] POST /identity/v3/users => generated 295 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:40.090509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.117298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c624a-ed14-4bc5-88f5-b18d4c1fba48 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.142586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.148206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.148206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.148206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.148206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.148206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.170923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 821/1633] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:30:40.176282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.182680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 813/1634] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:39 2022] PUT /identity/v3/projects/3a6664c5660f406ca1d0934b1cd54159/users/b3a0c9d02e5541faa993610cc4f6b900/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:40.185574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.191865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08018-d9c6-41e0-be33-2e3c04bf6048 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.289295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f70d4-9f62-403a-a5c0-3ea4080b85c2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.299961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.301087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.301374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.301652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.302442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.304106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.316426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 814/1635] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:40.321174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.327551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.328856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.329421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.329932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] PATH_INFO: `/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f/users/3d61dd1583e24979b3627c9968381ce4/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.338977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e46ada-ea77-480e-9c43-071f20715f9d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.346209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.347545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.347829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.348158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] PATH_INFO: `/v3/projects/3a6664c5660f406ca1d0934b1cd54159/users/b3a0c9d02e5541faa993610cc4f6b900/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.350495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6664c5660f406ca1d0934b1cd54159, user_id=b3a0c9d02e5541faa993610cc4f6b900, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.352561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5d8fc6b83a470b911fddbd798b6e7f, user_id=3d61dd1583e24979b3627c9968381ce4, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.353510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e990e5-e7c7-4925-b2ae-c4aa89724723 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.355795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.359276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:40.364484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 815/1636] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:40 2022] PUT /identity/v3/projects/3a6664c5660f406ca1d0934b1cd54159/users/b3a0c9d02e5541faa993610cc4f6b900/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:40.375084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73326-f952-424c-adba-663ddc85267f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.375394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73326-f952-424c-adba-663ddc85267f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.375708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73326-f952-424c-adba-663ddc85267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.402034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-08f73326-f952-424c-adba-663ddc85267f None None] MFA Rules not processed for user `b3a0c9d02e5541faa993610cc4f6b900`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:40.403245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-08f73326-f952-424c-adba-663ddc85267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:40.442301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:40.479257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 822/1637] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:40 2022] PUT /identity/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f/users/3d61dd1583e24979b3627c9968381ce4/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:40.484416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.494312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 816/1638] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:40.556027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57e7c-5ed8-4409-8138-b48f7338ea0d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.561893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.562585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.562924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.563213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.564281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.565925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c715c6f-e5ea-4f3c-8c17-23235f04622b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:40.579153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 823/1639] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:30:40.583502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73326-f952-424c-adba-663ddc85267f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:40.689098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73326-f952-424c-adba-663ddc85267f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:40.694403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:40.699413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:40.699712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:40.699999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] PATH_INFO: `/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f/users/3d61dd1583e24979b3627c9968381ce4/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:40.702175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5d8fc6b83a470b911fddbd798b6e7f, user_id=3d61dd1583e24979b3627c9968381ce4, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:40.706247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:41.191408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 817/1640] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:40 2022] PUT /identity/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f/users/3d61dd1583e24979b3627c9968381ce4/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] MFA Rules not processed for user `3d61dd1583e24979b3627c9968381ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:41.192152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 824/1641] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:41.249748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:41.365726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09675de5-4318-46ac-bbf1-46d33907de1c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:41.379552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:41.379855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:41.380254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:41.380652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:41.500982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:41.505261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:41.536895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:41.547833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 818/1642] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:41.565674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15dd2-bc44-4711-99e6-947bb2ac8a6a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:41.574488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:41.575504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:41.576290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:41.578240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:41.719224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:41.723023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:41.742747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 825/1643] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:30:42.135387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:42.145236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b448f8a-e92e-424e-9ad9-2a66fa0d40ae service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:42.149165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:42.149863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:42.151131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:42.151931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:42.167151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:42.223059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:42.225332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cc057-278a-443d-9d3d-2649d8ff4040 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:42.238218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 819/1644] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:42.729575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:42.783042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:42.891208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e75098-bdf7-4d31-9e21-6e78deea9c9b service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:42.905995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:42.906918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:42.907228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:42.907484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] PATH_INFO: `/v3/users/097d2584618a4292ac559e83a321669b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:42.910095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097d2584618a4292ac559e83a321669b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:42.911145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:42.911145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 14 14:30:42.911145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: Jul 14 14:30:42.912367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:43.218685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097d2584618a4292ac559e83a321669b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:43.219524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097d2584618a4292ac559e83a321669b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:43.290845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097d2584618a4292ac559e83a321669b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:43.293825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097d2584618a4292ac559e83a321669b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:43.322606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097d2584618a4292ac559e83a321669b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:43.355702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 826/1645] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:30:42 2022] DELETE /identity/v3/users/097d2584618a4292ac559e83a321669b => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:43.360680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:43.361633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:43.362136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:43.383366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:43.384600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:43.515446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 820/1646] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:43.555786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:43.558724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:43.699760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697f892-6749-4a10-8757-5b219ab71ca3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:43.710733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:43.714271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:43.714271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:43.714271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:43.734671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:43.736264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:43.758012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 827/1647] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:44.898649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:44.900180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:44.923220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d109-e509-4117-9f12-24edbd38b549 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:44.927192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:44.929906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:44.941719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:44.954267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:44.954267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:44.954267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] PATH_INFO: `/v3/users/4b08d1bedbc946868c9f9bc0b3808591` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:44.954267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b08d1bedbc946868c9f9bc0b3808591)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:44.962197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:44.983079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c544e-c0f7-46e1-ac4e-a30a3cfdc9a9 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:44.997910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:45.003878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.004867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.005171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] PATH_INFO: `/v3/projects/00aaa9fa0a0949188f8a80bf052d8105` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.007414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00aaa9fa0a0949188f8a80bf052d8105)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:45.011100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:45.208572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:45.243041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00aaa9fa0a0949188f8a80bf052d8105', 'request_id': 'req-d5c9273f-8151-410a-874a-784eb3e556e0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.257699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00aaa9fa0a0949188f8a80bf052d8105', 'request_id': 'req-d5c9273f-8151-410a-874a-784eb3e556e0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.261950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b08d1bedbc946868c9f9bc0b3808591'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.280043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b08d1bedbc946868c9f9bc0b3808591'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.351078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00aaa9fa0a0949188f8a80bf052d8105', 'request_id': 'req-d5c9273f-8151-410a-874a-784eb3e556e0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.370436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] The token cache is being invalidate because project 00aaa9fa0a0949188f8a80bf052d8105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:30:45.370984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.380771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d5c9273f-8151-410a-874a-784eb3e556e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.385131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 828/1648] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:30:44 2022] DELETE /identity/v3/projects/00aaa9fa0a0949188f8a80bf052d8105 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:45.398114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c556af-3439-4321-b1c9-d65e9dffbe68 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.399005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c556af-3439-4321-b1c9-d65e9dffbe68 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.399400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c556af-3439-4321-b1c9-d65e9dffbe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.440640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-84c556af-3439-4321-b1c9-d65e9dffbe68 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:45.443452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-84c556af-3439-4321-b1c9-d65e9dffbe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:45.448457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b08d1bedbc946868c9f9bc0b3808591'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.449446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b08d1bedbc946868c9f9bc0b3808591'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.450653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4b1c17ce-eb5c-428d-b47f-87df6484cd38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b08d1bedbc946868c9f9bc0b3808591'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:45.480756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 821/1649] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:30:44 2022] DELETE /identity/v3/users/4b08d1bedbc946868c9f9bc0b3808591 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:45.486340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.486941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.489319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.551659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:45.556447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:45.699304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:45.699406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 829/1650] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:45.699406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.699406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.699406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.709055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:45.710999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:45.717138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 822/1651] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:45.727402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:45.729454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:45.732429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 830/1652] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:45.736226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:45.754452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4e509-3c25-450c-93e0-574d3c7e22e9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:45.763496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:45.764577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.765509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.765932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.766865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:45.769248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:45.772435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:45.799398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 831/1653] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:30:45.801654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:45.817396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a952-6660-4d21-a15f-92100642964f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:45.823938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:45.824969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.825807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.826388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.827482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:45.830585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:45.846780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46fe8-2781-4f71-ad4f-ae8ea4035986 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:45.853609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:45.855040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:45.855335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:45.855677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:45.883295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:45.889510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:45.922525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 823/1654] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:46.399139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 832/1655] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:45 2022] POST /identity/v3/projects => generated 371 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:46.404003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:46.424001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd39b2-4834-4484-aab6-baad6b135b04 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:46.544956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:46.545837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.546129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.546573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:46.547308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:46.553057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:46.687145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 824/1656] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:46 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:46.692074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:46.709968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257179e-2eaa-483a-8ff9-53b3a03f7740 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:46.746445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:46.751054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.751351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.752520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:46.754735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:46.756494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:46.768357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 833/1657] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:46.772576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:46.784006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba14e-bf23-43fe-bd74-c7145d8f93cd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:46.789330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:46.790250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.790533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.791013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:46.791962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:46.793797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:46.804370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 825/1658] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:46.809940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:46.823759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b187-2c77-4735-8a3e-25743490093c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:46.829070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:46.830073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.830428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.830774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] PATH_INFO: `/v3/projects/aafcb1c3ea2a43458989304dde470035/users/e2289fcbe4474aa2893aa1c95495b13e/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:46.851609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafcb1c3ea2a43458989304dde470035, user_id=e2289fcbe4474aa2893aa1c95495b13e, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:46.856503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f17d1-e378-4497-b789-cf502cba9a57 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:46.890780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:46.892849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:46.920311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0e8d4-7277-40ab-b709-e457aab87792 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:46.932800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:46.936400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.937737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.938399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] PATH_INFO: `/v3/projects/beb09664df9647519b7a1ed5a30b3858` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:46.941014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb09664df9647519b7a1ed5a30b3858)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:46.943045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 14:30:46.943045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Jul 14 14:30:46.943045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: Jul 14 14:30:46.948547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:46.983662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 834/1659] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:46 2022] PUT /identity/v3/projects/aafcb1c3ea2a43458989304dde470035/users/e2289fcbe4474aa2893aa1c95495b13e/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:30:46.989821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:46.990114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:46.990401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.067628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] MFA Rules not processed for user `e2289fcbe4474aa2893aa1c95495b13e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:47.069981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:47.169900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb09664df9647519b7a1ed5a30b3858', 'request_id': 'req-a55a902e-4896-40fc-9331-93a5fc937468'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:47.202164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb09664df9647519b7a1ed5a30b3858', 'request_id': 'req-a55a902e-4896-40fc-9331-93a5fc937468'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:47.222620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb09664df9647519b7a1ed5a30b3858', 'request_id': 'req-a55a902e-4896-40fc-9331-93a5fc937468'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:47.247488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 835/1660] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:47.252211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:47.324444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] The token cache is being invalidate because project beb09664df9647519b7a1ed5a30b3858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:30:47.324741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:47.325980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a55a902e-4896-40fc-9331-93a5fc937468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:47.331158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 826/1661] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:30:46 2022] DELETE /identity/v3/projects/beb09664df9647519b7a1ed5a30b3858 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:47.336064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8625-f222-4690-8f91-603067dcd81a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.336364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8625-f222-4690-8f91-603067dcd81a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.336656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8625-f222-4690-8f91-603067dcd81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.388325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4aba8625-f222-4690-8f91-603067dcd81a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:47.393389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4aba8625-f222-4690-8f91-603067dcd81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:47.423062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de7dec-188e-4e52-877b-9e6f600d78a0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:47.504217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 827/1662] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:47.507330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.507767 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.508188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.531121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:47.532691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:47.551092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 828/1663] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:47.552570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:47.562583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:47.563434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.563732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.564015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.564731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:47.566600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54d53b-37b3-4bb1-93ba-c7b1f9e87802 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:47.566818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:47.573819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:47.574915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.575483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.576343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.577838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:47.578278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:47.581013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:47.613072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 829/1664] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:47.614463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:47.627649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc494e-fbb3-4942-9f46-17c7d5e94470 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:47.633931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:47.635011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.636091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.636458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.637186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:47.639227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:47.698381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 836/1665] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/projects => generated 373 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:47.702171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:47.704369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:47.801830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 830/1666] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:47.803161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:47.862879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47764-faa4-4ff2-8db9-1a8e8166865f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:47.962496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aefeb-350e-4ed8-890c-cfc4ba7017cd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:47.979047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:47.981027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:47.982433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:47.982823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:47.984976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:47.989405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.029509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.031348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.031721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.032080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.032876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.035069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.131564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 831/1667] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/users => generated 303 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:48.135747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.176243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780b80-c813-4d0e-80c8-d7ea7e4038a1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.189852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 837/1668] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:47 2022] POST /identity/v3/users => generated 293 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:48.194770 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.195097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.196256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.196641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.197024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.197919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.199695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.232926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e24c1-b6e0-4c15-b4fd-8cbc786cca20 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.256722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.257685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.260054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.260054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.260054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.260698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.289902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 832/1669] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:48.301873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.318748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:48.347792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 838/1670] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:48.352194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.359323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c490-4272-401f-b494-5176c9d9449b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.373541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.374602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.375415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.377013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.380931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.384246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.387082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5dd27-550a-4f3d-b542-2ff18a4e1253 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.401458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.402702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.403261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.404386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] PATH_INFO: `/v3/projects/35c664604dc644cfa813a5725a2421e5/users/906b411ba8064dfdb8b2af6ad7258ea2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.436269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 833/1671] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:48.437867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.458110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c664604dc644cfa813a5725a2421e5, user_id=906b411ba8064dfdb8b2af6ad7258ea2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.462844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851061-c679-45d8-865b-802dfe6b8aec admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.467107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.475129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.477335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.477705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.479978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] PATH_INFO: `/v3/projects/082f1a7c314d4590bf3e6f66e09dd9d0/users/d98e00d4e72241b5959fd37563477034/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.517737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] RBAC: Authorizing `identity:create_grant(project_id=082f1a7c314d4590bf3e6f66e09dd9d0, user_id=d98e00d4e72241b5959fd37563477034, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.522626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aadcf-7089-4c5d-a6ff-b18d474f183f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.579607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 839/1672] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:48 2022] PUT /identity/v3/projects/35c664604dc644cfa813a5725a2421e5/users/906b411ba8064dfdb8b2af6ad7258ea2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:48.582704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.602047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:48.695570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 834/1673] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:48 2022] PUT /identity/v3/projects/082f1a7c314d4590bf3e6f66e09dd9d0/users/d98e00d4e72241b5959fd37563477034/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:48.702738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.703182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.703568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.740493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99562ff-5fa8-43d8-88ec-7227c0214c29 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.745510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:48.750599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:48.862704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.863982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.864419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.864837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.865720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.867899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.887011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 840/1674] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:48.892799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.920279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6a1b-2f86-4809-81a0-1b8030c49319 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:48.925566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 835/1675] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:48.926087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:48.927338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.945854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.945854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] PATH_INFO: `/v3/projects/35c664604dc644cfa813a5725a2421e5/users/906b411ba8064dfdb8b2af6ad7258ea2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.945854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c664604dc644cfa813a5725a2421e5, user_id=906b411ba8064dfdb8b2af6ad7258ea2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:48.950216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9192f3-08a2-4795-bb6c-fc212003f5c2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:48.986529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:48.988291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 841/1676] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:48 2022] PUT /identity/v3/projects/35c664604dc644cfa813a5725a2421e5/users/906b411ba8064dfdb8b2af6ad7258ea2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:48.990268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:48.990566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:48.990852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:48.994169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:49.034325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] MFA Rules not processed for user `906b411ba8064dfdb8b2af6ad7258ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:49.036442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:49.186827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcb71d-540a-498d-a82e-578a2a750067 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:49.243797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 842/1677] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:49.261202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:49.269111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:49.336787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:49.342631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:49.343234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:49.343881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:49.399387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d340c3-a744-42a0-bb73-89d346c126c0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:49.411166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:49.412632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:49.412632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:49.412934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:49.436574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:49.438469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2aa8a-820a-4d6e-88dd-eb6b180d2377 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:49.474845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 843/1678] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:30:49.526090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:49.527688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b64124-8372-4de1-a975-1aece35b6604 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:49.563928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 836/1679] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:50.451196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:50.451737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:50.452187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:50.527422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] MFA Rules not processed for user `e2ac3e31fee448d7ae7a6d885f545636`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:50.527422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:50.678471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 844/1680] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:50.681037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17920df-a3b1-4007-81d8-4a91747dd86e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:50.681871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17920df-a3b1-4007-81d8-4a91747dd86e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:50.682896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17920df-a3b1-4007-81d8-4a91747dd86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:50.717784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e17920df-a3b1-4007-81d8-4a91747dd86e None None] MFA Rules not processed for user `e2ac3e31fee448d7ae7a6d885f545636`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:50.720375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e17920df-a3b1-4007-81d8-4a91747dd86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:50.747427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 837/1681] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:50.777930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:50.777930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:50.799736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:50.962637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3f77-bf9e-45cf-a91e-cebbfd89d14f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:50.970185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:50.971394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:50.971996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:50.972480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:50.982970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:50.986754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:51.003340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 845/1682] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:51.003781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557768d2-b24c-4e16-bf00-383898214b6c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.004074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557768d2-b24c-4e16-bf00-383898214b6c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.004956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557768d2-b24c-4e16-bf00-383898214b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.024396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:51.052676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-557768d2-b24c-4e16-bf00-383898214b6c None None] MFA Rules not processed for user `be6cba32a31a425e964d7222c3d6b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:51.054194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-557768d2-b24c-4e16-bf00-383898214b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.202141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 838/1683] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:51.216909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:51.219209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.220658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.222703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.223412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430600f-17d0-468f-8445-6250209fb56e service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:51.239611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:51.240858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.242141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.242796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] PATH_INFO: `/v3/users/e2289fcbe4474aa2893aa1c95495b13e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.244598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2289fcbe4474aa2893aa1c95495b13e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:51.247536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:51.272071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] MFA Rules not processed for user `be6cba32a31a425e964d7222c3d6b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:51.278156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.326453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 839/1684] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:51.343056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:51.345354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.409323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1227c0e-8bc5-4bd3-9c7a-ef3726fe02fe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:51.434919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:51.439353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.439919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.442831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.468227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:51.473310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:51.476593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2289fcbe4474aa2893aa1c95495b13e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:51.478291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2289fcbe4474aa2893aa1c95495b13e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:51.529393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 840/1685] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:51.531269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:51.539905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.564997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2289fcbe4474aa2893aa1c95495b13e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:51.565696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2289fcbe4474aa2893aa1c95495b13e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:51.578257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ed51df9e-631b-4b14-93c5-56713eebe354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2289fcbe4474aa2893aa1c95495b13e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:51.581041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522f566-49e8-4ff6-bd87-ea85304a9b6e service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:51.604879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:51.605701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.609488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.609899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.613246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.621348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 846/1686] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:30:51 2022] DELETE /identity/v3/users/e2289fcbe4474aa2893aa1c95495b13e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:51.625304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:51.630460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:51.630460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:51.677349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:51.680248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:51.800835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 847/1687] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:51.817804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:51.820507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462fc24-770a-4701-9334-658edaf53824 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:51.857027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 841/1688] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:51.938538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:51.940360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:52.034276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db9cd-ea8f-466d-a264-3b47e962ee49 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.034573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db9cd-ea8f-466d-a264-3b47e962ee49 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.035212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db9cd-ea8f-466d-a264-3b47e962ee49 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.037264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 842/1689] 10.222.0.93 () {54 vars in 912 bytes} [Thu Jul 14 14:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:52.046966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bff69-df89-41fb-8c64-138f52999cfa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.047551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bff69-df89-41fb-8c64-138f52999cfa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.048404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bff69-df89-41fb-8c64-138f52999cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.062545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f41ab-8a89-44eb-a904-9b4f5e516479 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:52.073802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:52.074975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.075330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.075762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.101747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:52.103379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:52.124425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 848/1690] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:52.207934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-244bff69-df89-41fb-8c64-138f52999cfa None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:52.214094 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-244bff69-df89-41fb-8c64-138f52999cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:52.255244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 843/1691] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 14 14:30:52.287381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:52.292038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:52.309095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faee82-62ff-4163-94df-8a57510c7598 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:52.321391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:52.322120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.322577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.322709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.353610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3c3fb-87f5-482d-af51-c62a6b3fbd9f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.353959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3c3fb-87f5-482d-af51-c62a6b3fbd9f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.354245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3c3fb-87f5-482d-af51-c62a6b3fbd9f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.355826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 844/1692] 10.222.0.93 () {54 vars in 912 bytes} [Thu Jul 14 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:52.364910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:52.367635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:52.370716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b2c9-dee4-489c-8912-38fa94eb16e5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.371889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b2c9-dee4-489c-8912-38fa94eb16e5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.372320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b2c9-dee4-489c-8912-38fa94eb16e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.413617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 849/1693] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:52.424978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ca23b2c9-dee4-489c-8912-38fa94eb16e5 None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:52.441492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ca23b2c9-dee4-489c-8912-38fa94eb16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:52.452042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 845/1694] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:52.517363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:52.531085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93444-b094-4011-88b8-38530ce318e9 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:52.542870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:52.543792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:52.547215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:52.547658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:52.568418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:52.573208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcda2e-a4bb-4c17-9a3d-30ad52256986 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:52.585875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:52.643465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 850/1695] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:53.686154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b594b-38d1-4f8e-9981-2a363cad9374 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:53.686973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b594b-38d1-4f8e-9981-2a363cad9374 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:53.690533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b594b-38d1-4f8e-9981-2a363cad9374 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:53.690533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 846/1696] 10.222.0.93 () {54 vars in 960 bytes} [Thu Jul 14 14:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:30:53.695000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:53.696183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:53.696620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:53.775108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:53.776416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:53.810643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:53.935396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:53.935906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:53.936197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.006744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 851/1697] 10.222.0.93 () {58 vars in 1057 bytes} [Thu Jul 14 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:54.008152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:54.040722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faaeb5-983f-4cd7-b71c-d90d79d29f32 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:54.052146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] MFA Rules not processed for user `d98e00d4e72241b5959fd37563477034`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:54.052666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:54.053527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.053879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.054163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.054407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.082027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.125370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:54.129474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:54.168000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 852/1698] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:54.169072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:54.174760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.219399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8213cce-dd8a-4285-81dc-56d3bbbdbf14 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:54.235053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:54.235689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.235951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.236232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.246894 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 847/1699] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:54.247879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:54.250531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.261375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:54.263596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c481-8e38-4527-a670-d0d0bd12e596 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:54.280528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d74f5-09bf-47a7-99ed-4d2eeec99f8d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:54.294965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 853/1700] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:54.298803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:54.299709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.300592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.302529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.303474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:54.306621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:54.318948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:54.368207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.368799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.369281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.447066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] MFA Rules not processed for user `be6cba32a31a425e964d7222c3d6b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:54.449663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.500326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 848/1701] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:54 2022] POST /identity/v3/projects => generated 405 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:54.503604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:54.517098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:54.615117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 854/1702] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:30:54.616118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:54.644882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9da1f-808a-40a1-bd30-3f8b3235a4bb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:54.684641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be352b6-9bff-4dd1-a874-0f2c4a0a740c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:54.729440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:54.746485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.746485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.746485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] PATH_INFO: `/v3/projects/aafcb1c3ea2a43458989304dde470035` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.749513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafcb1c3ea2a43458989304dde470035)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:54.751700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:54.767481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:54.768187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.768684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.768983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.770443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:54.772403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6feb2-9680-4c1b-aefa-bf5c453ae8f0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:54.923686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 855/1703] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:54 2022] POST /identity/v3/users => generated 309 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:54.928346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:54.928658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:54.928976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:54.986769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] MFA Rules not processed for user `bc26c3a3096b4b318d18d4c919f7a60a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:54.988168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:55.073417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafcb1c3ea2a43458989304dde470035', 'request_id': 'req-22d118b8-622d-47d8-8088-5322591c4353'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.090852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafcb1c3ea2a43458989304dde470035', 'request_id': 'req-22d118b8-622d-47d8-8088-5322591c4353'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.101896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafcb1c3ea2a43458989304dde470035', 'request_id': 'req-22d118b8-622d-47d8-8088-5322591c4353'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.159237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 856/1704] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:55.159237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:55.168663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:55.176403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55f32a-38f3-40f2-9990-61ced4a6de8a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:55.240901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] The token cache is being invalidate because project aafcb1c3ea2a43458989304dde470035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:30:55.241527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.242418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.246609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 849/1705] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:30:54 2022] DELETE /identity/v3/projects/aafcb1c3ea2a43458989304dde470035 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:55.250433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:55.261024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:55.311248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:55.312268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:55.312732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:55.313653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:55.315283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:55.317403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:55.333737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 857/1706] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:55.342270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:55.345028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:55.372945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7e181-b064-491f-873e-834e6fd80f5a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:55.408575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:55.411155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:55.411155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:55.411284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] PATH_INFO: `/v3/projects/55622f564d974755a6c2874f451187c5/users/8d2fc51a05124035b26992056cb4d164/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:55.458108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55622f564d974755a6c2874f451187c5, user_id=8d2fc51a05124035b26992056cb4d164, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:55.469266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:55.475751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d118b8-622d-47d8-8088-5322591c4353 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:55.490955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8064-fb56-4319-9458-4032541667df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:55.491904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:55.492697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8064-fb56-4319-9458-4032541667df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:55.493065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8064-fb56-4319-9458-4032541667df admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:55.493273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8064-fb56-4319-9458-4032541667df admin admin] PATH_INFO: `/v3/users/906b411ba8064dfdb8b2af6ad7258ea2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:55.496151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a8064-fb56-4319-9458-4032541667df admin admin] RBAC: Authorizing `identity:delete_user(user_id=906b411ba8064dfdb8b2af6ad7258ea2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:55.500441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a8064-fb56-4319-9458-4032541667df admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:55.678445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 858/1707] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:55 2022] PUT /identity/v3/projects/55622f564d974755a6c2874f451187c5/users/8d2fc51a05124035b26992056cb4d164/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:55.679371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:55.684496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:55.840383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af4fd-a80b-42e6-aa5e-122cf9332ec1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:55.848833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:55.849658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:55.849947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:55.850327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:55.852611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:55.926317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906b411ba8064dfdb8b2af6ad7258ea2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.940147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:55.941810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:55.948885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906b411ba8064dfdb8b2af6ad7258ea2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:55.970310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 859/1708] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:30:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:30:55.971679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:56.053583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906b411ba8064dfdb8b2af6ad7258ea2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:56.054600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906b411ba8064dfdb8b2af6ad7258ea2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:56.055493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906b411ba8064dfdb8b2af6ad7258ea2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:56.102749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 850/1709] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:30:55 2022] DELETE /identity/v3/users/906b411ba8064dfdb8b2af6ad7258ea2 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:56.114088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7a5ff-ca5d-46e3-b0a2-48d32a68b8e3 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:56.325455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:56.326200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:56.326487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:56.332875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:56.335512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:56.337345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:56.372963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 860/1710] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:56.374696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:56.377215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:56.426616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8064-fb56-4319-9458-4032541667df admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:56.448877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:56.454127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:56.454127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:56.457166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] PATH_INFO: `/v3/projects/55622f564d974755a6c2874f451187c5/users/8d2fc51a05124035b26992056cb4d164/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:56.463624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55622f564d974755a6c2874f451187c5, user_id=8d2fc51a05124035b26992056cb4d164, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:56.487729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e593b6-66f5-4332-96da-97e27d2ba0e5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:56.516376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 851/1711] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:56 2022] PUT /identity/v3/projects/55622f564d974755a6c2874f451187c5/users/8d2fc51a05124035b26992056cb4d164/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:56.874086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:57.385529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:57.387502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:57.420164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea986a8-a43f-428f-ae54-8016ed0614ed admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:57.592887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:57.594763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:57.595060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:57.595346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] PATH_INFO: `/v3/projects/35c664604dc644cfa813a5725a2421e5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:57.604346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c664604dc644cfa813a5725a2421e5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:57.606815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:57.849525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c664604dc644cfa813a5725a2421e5', 'request_id': 'req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:57.870663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c664604dc644cfa813a5725a2421e5', 'request_id': 'req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:57.993085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c664604dc644cfa813a5725a2421e5', 'request_id': 'req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:58.009076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] The token cache is being invalidate because project 35c664604dc644cfa813a5725a2421e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:30:58.009461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:58.010345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08c3cf05-0ae2-4505-b1ca-cec3cb0f99b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:30:58.016628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 861/1712] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:30:57 2022] DELETE /identity/v3/projects/35c664604dc644cfa813a5725a2421e5 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:58.018643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.019174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.019261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.045697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:58.046991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:58.156319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 852/1713] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:58.164267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.164562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.164843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.184679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:58.185929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:58.193802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 862/1714] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:58.200150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.209713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc923c-69ac-41bd-8a8f-80c978035079 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.214706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.215365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.215648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.215921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.216644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:30:58.216950 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.218672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.231621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 853/1715] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:30:58.235854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.245817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbedd6-c574-4fee-9b5a-930113c11ef7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.250496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.251789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.251789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.251956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.252836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.254828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.360274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 863/1716] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:58 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:30:58.361172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.373974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912f2-2968-4603-aa5a-f8034cd0f356 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.471067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.472073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.472548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.473017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.473990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.476243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.605869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 854/1717] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:58 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:30:58.610926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.641824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bb321-4285-4c46-ab41-7033fd547076 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.656173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.657480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.657978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.658432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.659581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.661913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.676661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 864/1718] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:58.680700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.697107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d9e61-f7a2-4cf8-90f4-48f40d8ffdd4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.703362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.704982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.705579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.706132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.707026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.708791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80880497-bab0-42f8-86d8-25fb804c47f1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.721941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 855/1719] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:58.737733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:58.754539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d7135-92d3-4021-92ce-d7047fd432d2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:58.761351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:58.762567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.763039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.763479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] PATH_INFO: `/v3/projects/94ae06149c3948f4a44c1650a176cac2/users/87d74557178641129c70c70e9e2da21f/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.792767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ae06149c3948f4a44c1650a176cac2, user_id=87d74557178641129c70c70e9e2da21f, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:58.796164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:58.902992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 865/1720] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:58 2022] PUT /identity/v3/projects/94ae06149c3948f4a44c1650a176cac2/users/87d74557178641129c70c70e9e2da21f/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:58.915784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb463-1c0f-456b-9690-f6adcea5c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:58.916601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb463-1c0f-456b-9690-f6adcea5c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:58.917087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb463-1c0f-456b-9690-f6adcea5c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:58.948541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:30:58.959620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3fddb463-1c0f-456b-9690-f6adcea5c0b4 None None] MFA Rules not processed for user `87d74557178641129c70c70e9e2da21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:58.961316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3fddb463-1c0f-456b-9690-f6adcea5c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:59.065758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 856/1721] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:59.068990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:59.098961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:59.100738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:59.101181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:59.151957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec9e-bd13-4d52-8e0d-233f43de6a45 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:59.164963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:59.164963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:59.164963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:59.164963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:59.165387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:59.165698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:59.177183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] MFA Rules not processed for user `8d2fc51a05124035b26992056cb4d164`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:30:59.185142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:30:59.332641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 866/1722] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:30:59 2022] POST /identity/v3/projects => generated 363 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:59.351317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:59.354690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 857/1723] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:30:59.498474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eceb23-9eb0-44be-93df-40f8dc70228e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:59.508444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:59.508444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:59.508444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:59.508444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:59.509138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:59.512935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:59.625554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 867/1724] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:30:59 2022] POST /identity/v3/users => generated 288 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:30:59.630726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:59.644210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d3827-9e26-41b2-8096-69b189d1082f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:59.649713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:59.652088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:59.652624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:59.654175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:59.655300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:59.656999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:59.671628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 858/1725] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:30:59.672861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:59.695784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafda686-3473-4215-b10f-8832680e1139 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:30:59.702090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:30:59.703097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:30:59.703436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:30:59.703886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] PATH_INFO: `/v3/projects/c49d68a62ef145a5a2bd2129b567b791/users/929559b032b448028461fd2543964763/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:30:59.727865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d68a62ef145a5a2bd2129b567b791, user_id=929559b032b448028461fd2543964763, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:30:59.731034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:30:59.904295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 868/1726] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:30:59 2022] PUT /identity/v3/projects/c49d68a62ef145a5a2bd2129b567b791/users/929559b032b448028461fd2543964763/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:30:59.908341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:30:59.921653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752303-e436-4105-b878-3594592e4d88 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:00.054703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:00.058018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:00.058018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:00.058018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:00.058353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:00.060222 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986ff43-e640-4fe0-8788-89660bf8f806 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:00.075771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 859/1727] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:00.079916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:00.104475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d1690-35fc-4d4e-80d8-6251e45e0fbc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:00.134543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:00.135532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:00.135822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:00.136106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] PATH_INFO: `/v3/projects/c49d68a62ef145a5a2bd2129b567b791/users/929559b032b448028461fd2543964763/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:00.140037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d68a62ef145a5a2bd2129b567b791, user_id=929559b032b448028461fd2543964763, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:00.142772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fc83-f1ac-419d-afd0-9e8ff1c2bcb7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:00.170675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 869/1728] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:00 2022] PUT /identity/v3/projects/c49d68a62ef145a5a2bd2129b567b791/users/929559b032b448028461fd2543964763/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:00.185693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:00.187610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:00.187610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:00.211235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:00.267942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] MFA Rules not processed for user `929559b032b448028461fd2543964763`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:00.269809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:00.362638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:00.362956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:00.363322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:00.427285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] MFA Rules not processed for user `bc26c3a3096b4b318d18d4c919f7a60a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:00.429234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:00.460110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 860/1729] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:00.477938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:00.481558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:00.633987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661202-02ea-46e1-8ae8-5bf6e728b6b3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:00.638171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 870/1730] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:00.645210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:00.645210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:00.645210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:00.645210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:00.669954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:00.669954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:00.688176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 861/1731] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:00.696953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:01.004848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:01.005397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:01.117011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc132e-3609-41a3-938c-0ff94e200a95 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:01.139154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:01.139154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.139154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.139154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] PATH_INFO: `/v3/users/b209aeb321f64346a6d85d6ebebdd637` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.139729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b209aeb321f64346a6d85d6ebebdd637)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:01.140879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:01.195340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:01.214733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e755-c077-4757-b697-76e752b19f62 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:01.225589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:01.226280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.226573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.241110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.365688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:01.368201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b5bdd-f260-4aba-861e-74e98dbfaa12 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:01.381138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 862/1732] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:01.382909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f64a9-86ea-4eb1-bc8c-2c0e62757667 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.383499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f64a9-86ea-4eb1-bc8c-2c0e62757667 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.383961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f64a9-86ea-4eb1-bc8c-2c0e62757667 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.386320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 863/1733] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:31:01.429487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.433561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.433561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.476145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:01.477538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:01.564521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 864/1734] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:01.581506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:01.583059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:01.617120 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0afab-077f-4905-88a2-30362fe0e095 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:01.627100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:01.628033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.628468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.630135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.648074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:01.654282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd18a-99a0-4cd7-b7ca-b7c4dcb42177 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:01.663774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b209aeb321f64346a6d85d6ebebdd637'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:01.665618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b209aeb321f64346a6d85d6ebebdd637'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:01.693516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 865/1735] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:01.820917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b209aeb321f64346a6d85d6ebebdd637'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:01.825122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b209aeb321f64346a6d85d6ebebdd637'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:01.835260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b209aeb321f64346a6d85d6ebebdd637'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:01.871480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 871/1736] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:01 2022] DELETE /identity/v3/users/b209aeb321f64346a6d85d6ebebdd637 => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:01.876488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb36e-7c99-4d86-b442-8ee414cb2194 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:01.883660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb36e-7c99-4d86-b442-8ee414cb2194 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:01.883660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb36e-7c99-4d86-b442-8ee414cb2194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:01.899918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b13cb36e-7c99-4d86-b442-8ee414cb2194 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:01.906127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b13cb36e-7c99-4d86-b442-8ee414cb2194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:02.040511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 866/1737] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:02.085512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:02.088317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:02.169641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:02.243070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e804-a6e4-4075-ba02-5a886f54c310 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:02.269380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:02.276591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:02.276766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:02.276993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:02.323844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:02.326906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88cf4-ae5c-4b0b-8f87-cb35ad3c70fd service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:02.361325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 872/1738] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:03.067328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610a1d9-8c5b-467d-a27a-f475fcb5131c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.067699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610a1d9-8c5b-467d-a27a-f475fcb5131c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.069156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610a1d9-8c5b-467d-a27a-f475fcb5131c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.155081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0610a1d9-8c5b-467d-a27a-f475fcb5131c None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:03.156090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0610a1d9-8c5b-467d-a27a-f475fcb5131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:03.257464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 867/1739] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:03.420081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e86432-8464-46de-90ea-686201b0b988 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.421087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e86432-8464-46de-90ea-686201b0b988 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.422595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e86432-8464-46de-90ea-686201b0b988 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.428940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 873/1740] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:31:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:31:03.438447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.439494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.439644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.450685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:03.475620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] MFA Rules not processed for user `768f4ab96bcf430f8e26b4d08d49581a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:03.477359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:03.549506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 868/1741] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:03.561415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e86432-8464-46de-90ea-686201b0b988 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:03.563632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-05e86432-8464-46de-90ea-686201b0b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:03.586797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e86432-8464-46de-90ea-686201b0b988 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:03.593186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:03.593894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.594473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.594767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.604394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:03.606066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:03.618225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 874/1742] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:03.628077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:03.629338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:03.648946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c52e39-df2e-4c1e-9863-ebac9b40c06d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:03.655159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:03.656616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.657153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.657636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] PATH_INFO: `/v3/users/d98e00d4e72241b5959fd37563477034` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.659698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98e00d4e72241b5959fd37563477034)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:03.662061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:03.817113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:03.819137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:03.967538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637e613-30ab-420b-b821-2cd5acf64c9e service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:03.977360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:03.979126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:03.979644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:03.980212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:03.994369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:03.997065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:04.012660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 875/1743] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:31:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 14:31:04.014044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:04.017747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:04.037290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c011be-9c37-48fd-b931-b4f99da37816 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:04.044847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:04.046780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:04.046780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:04.046780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] PATH_INFO: `/v3/projects/a344d7e2cadb46129ede551661428bc7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:04.050903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a344d7e2cadb46129ede551661428bc7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:04.053339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:04.063198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98e00d4e72241b5959fd37563477034'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.090547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98e00d4e72241b5959fd37563477034'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.230524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98e00d4e72241b5959fd37563477034'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.231693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98e00d4e72241b5959fd37563477034'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.232714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-882eef5d-0bb5-44b6-87d0-be4262e3772a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98e00d4e72241b5959fd37563477034'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.263770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 869/1744] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:03 2022] DELETE /identity/v3/users/d98e00d4e72241b5959fd37563477034 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:04.276887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a344d7e2cadb46129ede551661428bc7', 'request_id': 'req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.289876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a344d7e2cadb46129ede551661428bc7', 'request_id': 'req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.433031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a344d7e2cadb46129ede551661428bc7', 'request_id': 'req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.453447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] The token cache is being invalidate because project a344d7e2cadb46129ede551661428bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:04.453748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.478404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e63b85dc-853c-4ab0-84ad-22d0c3c67ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:04.488014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 876/1745] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:04 2022] DELETE /identity/v3/projects/a344d7e2cadb46129ede551661428bc7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:04.493352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:04.493754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:04.493962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:04.596182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:04.597690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:04.600100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:04.779074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:04.779757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:04.780205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 870/1746] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:04.780520 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:04.858818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:04.867007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:04.898317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 877/1747] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:04.899348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:04.935697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31601359-1da9-45a7-b428-eac27b401ee1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:04.942389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:04.943190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:04.943474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:04.943904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:04.944861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:04.945153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:04.946937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:04.970054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:04.971195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 871/1748] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:04.985655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef1f3c-a32f-4795-8ba8-5b66e573bb69 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.021527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.022941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.023649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.024251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.025173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.030539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.248984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:05.260123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 878/1749] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:04 2022] POST /identity/v3/projects => generated 355 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:05.266485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d2e36-8277-477c-bb27-0a99ebc39207 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.312275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:05.314331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:05.428633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.429464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.429988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.431237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.432263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.438567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.512251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eda0c-800f-4c33-8f2f-4c9408702759 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.517906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.518810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.519107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.519406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] PATH_INFO: `/v3/projects/082f1a7c314d4590bf3e6f66e09dd9d0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.521332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082f1a7c314d4590bf3e6f66e09dd9d0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.523376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.693303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 872/1750] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:05 2022] POST /identity/v3/users => generated 278 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:05.698149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:05.708783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d141c-71c0-4831-ab22-7266d8766d13 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.714533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.715340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.715703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.716090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.716951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.719751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.737625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 873/1751] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:05.744681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:05.751388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bb21-8a5f-4a3d-be0d-e212f59e165b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.756639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.758278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.758575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.758860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.759616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.762478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.782113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 874/1752] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:05.789901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:05.809905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ada62-0332-413b-a5dc-eba8bad6e257 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:05.818084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:05.825025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.825359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.826827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] PATH_INFO: `/v3/projects/5f01694d5df04a6a86bcec0c94abc902/users/6de72a8e5c494b318dad785f397fe5ba/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.855771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082f1a7c314d4590bf3e6f66e09dd9d0', 'request_id': 'req-178b1304-1569-4a2d-8621-1eeccc1b5520'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:05.862518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f01694d5df04a6a86bcec0c94abc902, user_id=6de72a8e5c494b318dad785f397fe5ba, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:05.865462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afd31a-ddc1-4761-a7b6-aac5f5dd3b85 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:05.873376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082f1a7c314d4590bf3e6f66e09dd9d0', 'request_id': 'req-178b1304-1569-4a2d-8621-1eeccc1b5520'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:05.960542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 875/1753] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:05 2022] PUT /identity/v3/projects/5f01694d5df04a6a86bcec0c94abc902/users/6de72a8e5c494b318dad785f397fe5ba/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:05.979500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:05.979833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:05.980076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:05.998876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082f1a7c314d4590bf3e6f66e09dd9d0', 'request_id': 'req-178b1304-1569-4a2d-8621-1eeccc1b5520'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.024946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] The token cache is being invalidate because project 082f1a7c314d4590bf3e6f66e09dd9d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:06.025545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.026261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.028191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 879/1754] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:05 2022] DELETE /identity/v3/projects/082f1a7c314d4590bf3e6f66e09dd9d0 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:06.032939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:06.036507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:06.039506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] MFA Rules not processed for user `6de72a8e5c494b318dad785f397fe5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:06.040919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:06.207807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 876/1755] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:06.208029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b1304-1569-4a2d-8621-1eeccc1b5520 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:06.209301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:06.211433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:06.336963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:06.336963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:06.337538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:06.337639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] PATH_INFO: `/v3/users/8d2fc51a05124035b26992056cb4d164` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:06.339150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2fc51a05124035b26992056cb4d164)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:06.351583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:06.389405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda5994-77dd-4501-8e64-43c9af251381 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:06.414396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:06.415300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:06.415775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:06.416887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:06.417810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:06.420079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:06.530545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 877/1756] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:06 2022] POST /identity/v3/projects => generated 357 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:06.536319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:06.570801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2fc51a05124035b26992056cb4d164'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.572174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2fc51a05124035b26992056cb4d164'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.625591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb30a2-6e42-40cc-91ed-7b7383974837 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:06.630500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:06.634373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:06.634373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:06.634373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:06.634373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:06.634714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:06.670189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2fc51a05124035b26992056cb4d164'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.670844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2fc51a05124035b26992056cb4d164'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.679272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2fc51a05124035b26992056cb4d164'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:06.705158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 880/1757] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:06 2022] DELETE /identity/v3/users/8d2fc51a05124035b26992056cb4d164 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:06.753115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 878/1758] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:06 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:06.754405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:06.755880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:06.855636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ecacb-a047-4e62-bb85-9579355d3f87 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:06.864348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:06.864348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:06.864348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:06.864348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:06.864937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:06.866510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:06.882679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 881/1759] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:06.883358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:06.902791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca51b-6cc4-4277-9793-e1567cddc06a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:06.925251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:06.930785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:06.931220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:06.931689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] PATH_INFO: `/v3/projects/95af44109a354087bfcc33425aba8415/users/52936cb1980e4728868f5614b5edefd0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:06.969136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95af44109a354087bfcc33425aba8415, user_id=52936cb1980e4728868f5614b5edefd0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:06.972998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.072891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.073427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 879/1760] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:06 2022] PUT /identity/v3/projects/95af44109a354087bfcc33425aba8415/users/52936cb1980e4728868f5614b5edefd0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:07.089313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593a267-b2a8-42de-8b4c-8226451818a0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.109831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.116717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.213924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.213924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.213924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.213924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.214305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.214305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.227427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 882/1761] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:07.232619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.244296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857503-7d4c-4794-9789-d3c16f9d5323 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.245755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45dd6e-6982-401f-ae70-5194b46956fa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.257152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.258253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.258616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.258973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] PATH_INFO: `/v3/projects/95af44109a354087bfcc33425aba8415/users/52936cb1980e4728868f5614b5edefd0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.268846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.271684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.272781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.278824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] PATH_INFO: `/v3/projects/55622f564d974755a6c2874f451187c5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.278824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55622f564d974755a6c2874f451187c5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.279100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95af44109a354087bfcc33425aba8415, user_id=52936cb1980e4728868f5614b5edefd0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.281557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9c51f-e248-47f6-916b-51e926993f41 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.283338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.313136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 883/1762] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:07 2022] PUT /identity/v3/projects/95af44109a354087bfcc33425aba8415/users/52936cb1980e4728868f5614b5edefd0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:07.322741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752625a-25c5-4901-819f-f855a2305dec None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.323041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752625a-25c5-4901-819f-f855a2305dec None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.323334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752625a-25c5-4901-819f-f855a2305dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.367141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4752625a-25c5-4901-819f-f855a2305dec None None] MFA Rules not processed for user `52936cb1980e4728868f5614b5edefd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:07.368816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4752625a-25c5-4901-819f-f855a2305dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.451794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55622f564d974755a6c2874f451187c5', 'request_id': 'req-fe501f83-9cc5-4861-a9fa-799a0dead48e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:07.463460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55622f564d974755a6c2874f451187c5', 'request_id': 'req-fe501f83-9cc5-4861-a9fa-799a0dead48e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:07.488362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55622f564d974755a6c2874f451187c5', 'request_id': 'req-fe501f83-9cc5-4861-a9fa-799a0dead48e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:07.506961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 884/1763] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:07.538667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752625a-25c5-4901-819f-f855a2305dec None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.540757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4752625a-25c5-4901-819f-f855a2305dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.547645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:07.613146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] The token cache is being invalidate because project 55622f564d974755a6c2874f451187c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:07.613905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:07.616180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe501f83-9cc5-4861-a9fa-799a0dead48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:07.619948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 880/1764] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:07 2022] DELETE /identity/v3/projects/55622f564d974755a6c2874f451187c5 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:07.623436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752625a-25c5-4901-819f-f855a2305dec None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.625706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb089d2-1f2b-4d96-aa73-29e2c4b54102 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.626153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb089d2-1f2b-4d96-aa73-29e2c4b54102 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.626605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb089d2-1f2b-4d96-aa73-29e2c4b54102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.656786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ceb089d2-1f2b-4d96-aa73-29e2c4b54102 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:07.658432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ceb089d2-1f2b-4d96-aa73-29e2c4b54102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.707930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.708762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.709119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.709457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.712303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.785600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 881/1765] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:07.795469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:07.796720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.797023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.797307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.822891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:07.824116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.841664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.841664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e71b-877b-4cb3-80a2-f95812643253 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.868658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 882/1766] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:07.880819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.881925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 885/1767] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:07 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:07.883925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b12f-a20d-4123-a342-01852a446628 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.884963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b12f-a20d-4123-a342-01852a446628 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.885252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b12f-a20d-4123-a342-01852a446628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.889284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc1846-383e-476f-95b0-038af208fa0c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.898819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.900293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.900618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.900899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.903451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:07.906295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.910114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:07.920769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f875b12f-a20d-4123-a342-01852a446628 None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:07.920769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f875b12f-a20d-4123-a342-01852a446628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:07.925163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 883/1768] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:07.931261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:07.946110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391518a7-b2f9-4a96-9e75-fed9bb029a98 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:07.952138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:07.952897 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:07.953802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:07.954097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:07.955001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:07.957047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.030375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 886/1769] 10.222.0.91 () {58 vars in 1015 bytes} [Thu Jul 14 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:08.115112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 884/1770] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:07 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:08.117411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b12f-a20d-4123-a342-01852a446628 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.137331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b12f-a20d-4123-a342-01852a446628 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.270686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.273055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:08.293982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.297331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.297331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.297331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.297331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.298269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.348760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6fe92-6a83-4684-8fba-88a6a9480a26 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.353683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.354627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.354922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.355605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.365598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:08.462995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.468926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.479821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 885/1771] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:08.480512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.482462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:08.509622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 887/1772] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:08 2022] POST /identity/v3/users => generated 291 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:08.511155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.516814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:08.598294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86e3b-a16a-4e81-9d30-262f3bde5f88 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.610223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.613958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.613958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.613958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.627966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.629301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.654538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 888/1773] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:08.656451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4aba1-d549-4769-84fe-290a0bb0f7bd service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.664047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.665118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.665732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.666297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.672538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.676368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.678072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.690357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 886/1774] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:08.691248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:08.691620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.708919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52fa7-0d77-4f5d-8594-7e0e2cb62c99 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.721797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08c030-ef85-4e24-a069-38db72497f2f service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.727366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.727643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.728037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.728233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.728349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.728571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.728669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.728835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.730800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.733836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.744615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.746147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.750215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 887/1775] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:08.754705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.772702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 889/1776] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:08.775200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f755f5-1450-492d-863f-a918743a95e4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.783624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.784576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.784943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.785341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.786233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.788759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295822b-4b86-45fb-92cf-a23cc40bc7f2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.807691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 888/1777] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:08.811898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:08.826289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5516-1194-402e-8f2e-261ee4c387d5 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:08.831945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:08.833317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.833737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.834119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] PATH_INFO: `/v3/projects/63a7e5d3bb8545f1b6c3b92ba4889cad/users/01c91682c913412392874ea398100c7c/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:08.856064 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a7e5d3bb8545f1b6c3b92ba4889cad, user_id=01c91682c913412392874ea398100c7c, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:08.861132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:08.976305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:08.985124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 890/1778] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:08 2022] PUT /identity/v3/projects/63a7e5d3bb8545f1b6c3b92ba4889cad/users/01c91682c913412392874ea398100c7c/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:08.997753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:08.998074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:08.998359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:09.045241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] MFA Rules not processed for user `01c91682c913412392874ea398100c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:09.060510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:09.189826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 889/1779] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:09.193242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:09.333151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972c5d-d589-4ae5-a480-017e24cf9f8b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:09.337659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:09.339029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:09.339333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:09.339618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:09.340339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:09.342316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:09.514532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 891/1780] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:09 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:09.515195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:09.561921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98751be0-07a8-4994-80e9-e09838a2fc92 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:09.659919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:09.662585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:09.726220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:09.726220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:09.726220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:09.726220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:09.726578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:09.742955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:09.800496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2cd4e-f032-4f1e-b908-83500192d2c2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:09.819959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:09.821109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:09.821552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:09.823582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] PATH_INFO: `/v3/users/b83232101c954b88bc2dfb4c040c60b5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:09.825198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83232101c954b88bc2dfb4c040c60b5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:09.830842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:09.926326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 890/1781] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:09 2022] POST /identity/v3/users => generated 296 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:09.926326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:09.943340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43441d-fccf-4166-b299-e9b2185f7a2e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:09.951121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:09.952391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:09.953479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:09.953867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:09.955023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:09.961886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:10.005882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 891/1782] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:10.009857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:10.035933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a0716-4bce-42e3-826c-a2c25fde035e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:10.065901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:10.067186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.067186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.067582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] PATH_INFO: `/v3/projects/4a26d10b918c4d65bfc06075f630a9a2/users/4e95b5e5e2e14b47b31dd6002fc1b437/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.114170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a26d10b918c4d65bfc06075f630a9a2, user_id=4e95b5e5e2e14b47b31dd6002fc1b437, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:10.117657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:10.254129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83232101c954b88bc2dfb4c040c60b5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:10.255371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83232101c954b88bc2dfb4c040c60b5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:10.308989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 892/1783] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:10 2022] PUT /identity/v3/projects/4a26d10b918c4d65bfc06075f630a9a2/users/4e95b5e5e2e14b47b31dd6002fc1b437/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:10.313142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:10.317477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:10.419345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83232101c954b88bc2dfb4c040c60b5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:10.420463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83232101c954b88bc2dfb4c040c60b5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:10.434318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34ed7e64-5150-4fa5-9dd6-03879733504a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83232101c954b88bc2dfb4c040c60b5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:10.475070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 892/1784] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:09 2022] DELETE /identity/v3/users/b83232101c954b88bc2dfb4c040c60b5 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:10.485618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.486004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.486299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.520528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddc88d-d636-483c-b261-f18476077bc5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:10.527325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:10.529282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:10.731592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 893/1785] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:10.747052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:10.753390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:10.754494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.754787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:10.755158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.756881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.757786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:10.762482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:10.799996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 893/1786] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:10.802596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:10.833970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3187b-768a-43e0-937b-699dc8d3c046 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:10.839044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:10.839999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.840677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.840677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.843326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f98c-c19c-46b4-a022-528befeee937 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:10.859405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:10.861541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6410831-d70a-4aa6-aff2-38895615a3a4 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:10.875750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:10.876674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.877001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.877286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] PATH_INFO: `/v3/projects/4a26d10b918c4d65bfc06075f630a9a2/users/4e95b5e5e2e14b47b31dd6002fc1b437/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.883286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a26d10b918c4d65bfc06075f630a9a2, user_id=4e95b5e5e2e14b47b31dd6002fc1b437, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:10.886513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5baa5c3-8cf6-4017-bad3-fef7e77d0116 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:10.890812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 894/1787] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:10.935809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 894/1788] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:10 2022] PUT /identity/v3/projects/4a26d10b918c4d65bfc06075f630a9a2/users/4e95b5e5e2e14b47b31dd6002fc1b437/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:10.960677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.961088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.962397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.978223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5da1b-ba3d-4377-9402-6a1a43a8e8c2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.978582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5da1b-ba3d-4377-9402-6a1a43a8e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.984191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5da1b-ba3d-4377-9402-6a1a43a8e8c2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:10.984191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 895/1789] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:31:10.995595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:10.995884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:10.996169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.044214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] MFA Rules not processed for user `4e95b5e5e2e14b47b31dd6002fc1b437`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:11.045658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.046979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:11.048300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.069373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:11.185777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 895/1790] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:11.202799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:11.204359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.254786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 896/1791] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:11.309053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5e1e-6b09-400f-9752-141a1d3ff3ec None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:11.314207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:11.317116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:11.319203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.328952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.329928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.337523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.356733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821772e-f897-427c-b1f1-27d3766bb7eb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:11.364211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:11.368123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:11.375244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:11.375244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.375244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.375244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.396428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 896/1792] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:11.399470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:11.401718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:11.419420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 897/1793] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:11.597285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:11.617292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:11.618740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.719545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735c501-336d-4c32-9a95-836cc570715c service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:11.728344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:11.729962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.730325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.730569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] PATH_INFO: `/v3/users/87d74557178641129c70c70e9e2da21f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.732099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d74557178641129c70c70e9e2da21f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:11.734422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:11.755463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:11.757663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.776849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219931-ffe1-4821-ae1a-cd33db4d717a service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:11.780841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:11.784072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.784072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.784072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.784331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:11.846874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:11.848229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4425336-b0e1-4cf5-9a66-9c6dd922de98 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:11.858512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 898/1794] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:11.967960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33e39-5ef7-4cb1-87ac-95117a979add None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.968182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33e39-5ef7-4cb1-87ac-95117a979add None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.968465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33e39-5ef7-4cb1-87ac-95117a979add None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:11.969873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 899/1795] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:31:11.995521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:11.995521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:11.996079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:12.022106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:12.027403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:12.039493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 900/1796] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:31:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:12.076277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:12.095763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139a4c8-61ff-412c-9942-37e203580843 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:12.104003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:12.104864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:12.105298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:12.105745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:12.127722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:12.130226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968622c-28a8-40c4-96c9-987c0e68d3f6 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:12.136791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d74557178641129c70c70e9e2da21f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.138249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d74557178641129c70c70e9e2da21f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.143308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 901/1797] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:12.336733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d74557178641129c70c70e9e2da21f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.337670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d74557178641129c70c70e9e2da21f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.366893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d74557178641129c70c70e9e2da21f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.386643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 897/1798] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:11 2022] DELETE /identity/v3/users/87d74557178641129c70c70e9e2da21f => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:12.394053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:12.394244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:12.394494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:12.412585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:12.413843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:12.490466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 902/1799] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:12.511008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:12.512772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:12.615275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39d793-d574-4402-8e36-c388a8359fcd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:12.620689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:12.621464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:12.622035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:12.629503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:12.639996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:12.641659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:12.663431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:12.664979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:12.680746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 898/1800] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:12.704857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0c22-64b5-4fd5-b969-32facb98a056 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:12.712996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:12.714037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:12.714398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:12.715017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] PATH_INFO: `/v3/projects/fc023ab14cbc4b488105759e7aeb7e40` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:12.716424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc023ab14cbc4b488105759e7aeb7e40)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:12.718481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:12.888895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc023ab14cbc4b488105759e7aeb7e40', 'request_id': 'req-b0df336a-0ac4-4e08-8b55-d60776948c0e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.895269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc023ab14cbc4b488105759e7aeb7e40', 'request_id': 'req-b0df336a-0ac4-4e08-8b55-d60776948c0e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.901048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc023ab14cbc4b488105759e7aeb7e40', 'request_id': 'req-b0df336a-0ac4-4e08-8b55-d60776948c0e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.993022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] The token cache is being invalidate because project fc023ab14cbc4b488105759e7aeb7e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:12.993547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.995048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:12.997873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 903/1801] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:12 2022] DELETE /identity/v3/projects/fc023ab14cbc4b488105759e7aeb7e40 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:13.001845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:13.014859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed401d-2476-407f-9179-b396784cf913 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:13.099367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:13.101483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:13.102213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:13.110617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] PATH_INFO: `/v3/users/b3a0c9d02e5541faa993610cc4f6b900` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:13.110617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a0c9d02e5541faa993610cc4f6b900)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:13.110617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:13.421599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:13.423619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:13.427649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a0c9d02e5541faa993610cc4f6b900'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.427947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a0c9d02e5541faa993610cc4f6b900'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.439859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df336a-0ac4-4e08-8b55-d60776948c0e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:13.446517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:13.452994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:13.452994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:13.452994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] PATH_INFO: `/v3/projects/94ae06149c3948f4a44c1650a176cac2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:13.452994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ae06149c3948f4a44c1650a176cac2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:13.452994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:13.524519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a0c9d02e5541faa993610cc4f6b900'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.525277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a0c9d02e5541faa993610cc4f6b900'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.532973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a0c9d02e5541faa993610cc4f6b900'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.550727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 899/1802] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:13 2022] DELETE /identity/v3/users/b3a0c9d02e5541faa993610cc4f6b900 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:13.583857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ae06149c3948f4a44c1650a176cac2', 'request_id': 'req-0605de76-5386-4b3a-8ac2-683207f78343'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.590361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ae06149c3948f4a44c1650a176cac2', 'request_id': 'req-0605de76-5386-4b3a-8ac2-683207f78343'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.595954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ae06149c3948f4a44c1650a176cac2', 'request_id': 'req-0605de76-5386-4b3a-8ac2-683207f78343'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.792478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] The token cache is being invalidate because project 94ae06149c3948f4a44c1650a176cac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:13.792842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.793464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:13.795065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 904/1803] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:13 2022] DELETE /identity/v3/projects/94ae06149c3948f4a44c1650a176cac2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:13.799360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:13.800877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:13.871447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8998ff-fcac-41da-9652-851498ff3111 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:13.929603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:13.930668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:13.931063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:13.931465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] PATH_INFO: `/v3/users/929559b032b448028461fd2543964763` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:13.933319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=929559b032b448028461fd2543964763)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:13.935304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:14.197782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:14.199487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:14.222000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605de76-5386-4b3a-8ac2-683207f78343 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:14.227784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:14.229885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:14.230017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:14.230288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] PATH_INFO: `/v3/projects/3a6664c5660f406ca1d0934b1cd54159` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:14.232245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6664c5660f406ca1d0934b1cd54159)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:14.234560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:14.348820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929559b032b448028461fd2543964763'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.349654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929559b032b448028461fd2543964763'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.431470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6664c5660f406ca1d0934b1cd54159', 'request_id': 'req-d6da0038-0795-4736-92a4-a3a941312d92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.438886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929559b032b448028461fd2543964763'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.439446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929559b032b448028461fd2543964763'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.445964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6664c5660f406ca1d0934b1cd54159', 'request_id': 'req-d6da0038-0795-4736-92a4-a3a941312d92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.456107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6664c5660f406ca1d0934b1cd54159', 'request_id': 'req-d6da0038-0795-4736-92a4-a3a941312d92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.457371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb7b7560-3f82-44ee-98a7-0f5603353730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929559b032b448028461fd2543964763'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.510694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 900/1804] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:13 2022] DELETE /identity/v3/users/929559b032b448028461fd2543964763 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:14.623070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] The token cache is being invalidate because project 3a6664c5660f406ca1d0934b1cd54159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:14.623370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.624049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d6da0038-0795-4736-92a4-a3a941312d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:14.634044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 905/1805] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:14 2022] DELETE /identity/v3/projects/3a6664c5660f406ca1d0934b1cd54159 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:14.634485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:14.634940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:14.635285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:14.654790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:14.656266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:14.736172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:14.736172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:14.736782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:14.746013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 901/1806] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:14.763743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:14.765241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:14.782812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:14.789191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 906/1807] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:14.795636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e0f3-a3ed-40f0-b2ba-20d2dde0e106 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:14.808639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:14.809685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:14.810110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:14.810486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:14.811353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:14.811765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:14.813887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:14.838503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 902/1808] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:31:14.843125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:14.858568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa3b19-ad33-4350-970d-b610ee695229 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:14.865864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:14.870495 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:14.870943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:14.871329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:14.872274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:14.875386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:14.995476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 907/1809] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:14 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:14.999567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.011823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504cd16-72fd-43dd-ac2b-82d258d63781 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.142729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.144164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.146799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:15.147009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.147839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.148696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.149704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.153081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.180936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663a9dd-aeaa-4cf3-829d-9e134d51b428 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.188797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.189700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.190104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.190363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] PATH_INFO: `/v3/projects/c49d68a62ef145a5a2bd2129b567b791` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.191683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49d68a62ef145a5a2bd2129b567b791)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.193759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.265118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 903/1810] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:14 2022] POST /identity/v3/users => generated 283 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:15.271644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.291491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83815a82-88ed-4706-bea0-107403c4a147 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.313353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.313353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.313353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.313353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.318656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.323096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.353254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 904/1811] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:15.363210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.364779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49d68a62ef145a5a2bd2129b567b791', 'request_id': 'req-1224b62d-d712-40a7-a471-050ca6c752e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:15.380284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49d68a62ef145a5a2bd2129b567b791', 'request_id': 'req-1224b62d-d712-40a7-a471-050ca6c752e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:15.382528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba739a-7a05-4ccc-a47f-61db8e15d541 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.387631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49d68a62ef145a5a2bd2129b567b791', 'request_id': 'req-1224b62d-d712-40a7-a471-050ca6c752e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:15.468703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.469429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.469832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.470006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.470762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.472713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.486732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 905/1812] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:15.494256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.508428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f07a1-5d4d-45ff-9cfa-da9a5bd986de admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.513190 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.514136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.514427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.514707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] PATH_INFO: `/v3/projects/08aa621fa9914449a1a51066e25673eb/users/5977443511e246738c4467f93d8454d2/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.526351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] The token cache is being invalidate because project c49d68a62ef145a5a2bd2129b567b791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:15.527219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:15.528210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1224b62d-d712-40a7-a471-050ca6c752e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:15.529735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 908/1813] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:15 2022] DELETE /identity/v3/projects/c49d68a62ef145a5a2bd2129b567b791 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:15.536204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f9210-848b-45f4-81f0-19ce69a87668 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.536604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f9210-848b-45f4-81f0-19ce69a87668 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.536899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f9210-848b-45f4-81f0-19ce69a87668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.548783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08aa621fa9914449a1a51066e25673eb, user_id=5977443511e246738c4467f93d8454d2, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.551805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58009425-cfcf-488d-9439-071b5c2ccb73 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.573563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-389f9210-848b-45f4-81f0-19ce69a87668 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:15.579076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-389f9210-848b-45f4-81f0-19ce69a87668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:15.605543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 909/1814] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:15.613824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.617456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.617456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.650799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 906/1815] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:15 2022] PUT /identity/v3/projects/08aa621fa9914449a1a51066e25673eb/users/5977443511e246738c4467f93d8454d2/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:15.661669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.662222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.662729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.670865 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:15.672857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:15.698820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:15.760418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] MFA Rules not processed for user `5977443511e246738c4467f93d8454d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:15.763256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:15.882671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 910/1816] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:15.884262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.911368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7decad-60cf-406d-a3b8-ce2aa23a83a7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.933001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:15.933935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:15.934366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:15.934816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:15.935708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:15.936353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:15.939960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:15.965720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 907/1817] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:15.973123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.976852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 911/1818] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:15.976852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:15.977024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:15.996795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:15.999193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfc6ad-e310-406f-934c-29b7697e1862 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:15.999798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13a7d9-35ab-4736-820c-b4771ce7d164 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.005109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.006169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.006699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.007187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.008048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.010676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.013836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.014520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.014808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.015091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.015986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.018926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.146941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 908/1819] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:15 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:16.151480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.182260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 912/1820] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:15 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:16.186345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.295256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bac12-74a7-4ce5-8615-5a925e144e9b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.297343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b45ab0-c405-4401-8b69-85000044e591 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.330799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.330799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.330799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.330799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.331078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.331152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.331490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.334218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.340240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.341055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.341148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.341148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.444271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 909/1821] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:16 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:16.454451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.469644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 913/1822] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:16 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:16.471864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.481495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0077a16-651a-4a3b-9d06-81d065309d90 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.490214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.490214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.490214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.490681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.491411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.493291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.504235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685c4d2-39fe-4409-8ca3-965882c87499 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.513823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 910/1823] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:16.519728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.521586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.525075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.526340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.527536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.529677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.537424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.551123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803577-127a-40a5-b54e-09a9e49c6842 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.568354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.569604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.571809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 914/1824] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:16.573359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.573945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] PATH_INFO: `/v3/projects/8647fb57a7704a438e6041d97462f797/users/47b5e0062da841f4b3a07df7705c04c0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.576650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.596868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8647fb57a7704a438e6041d97462f797, user_id=47b5e0062da841f4b3a07df7705c04c0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.602752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.611231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967f496-d6e8-4ab1-8594-de0efb252cf5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.625959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.626652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.627130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.627246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.628381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.635365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.679892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 915/1825] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:16.681062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.708171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb130a1-018a-4bd4-9726-d3abbe2531c9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.713899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 911/1826] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:16 2022] PUT /identity/v3/projects/8647fb57a7704a438e6041d97462f797/users/47b5e0062da841f4b3a07df7705c04c0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:16.716889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:16.891587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.894289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.894660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.895015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] PATH_INFO: `/v3/projects/bfb5cccc3e88497a96eb3c780e4d04fa/users/fa658a8b2ac94bc2b6fe1c8366657c8a/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.928929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed9eb-6a06-4e32-9793-475ab53099e3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:16.942252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:16.946541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:16.948138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:16.948831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:16.951597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.957066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.958202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb5cccc3e88497a96eb3c780e4d04fa, user_id=fa658a8b2ac94bc2b6fe1c8366657c8a, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:16.963401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da46a-b93c-4c3d-bc75-b46f35b7313e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:16.993605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 912/1827] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:16.995510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:17.022614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed233c-6101-46f0-aede-aeedcf5074ac admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:17.030673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:17.032222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:17.032535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:17.032826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] PATH_INFO: `/v3/projects/8647fb57a7704a438e6041d97462f797/users/47b5e0062da841f4b3a07df7705c04c0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:17.035604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8647fb57a7704a438e6041d97462f797, user_id=47b5e0062da841f4b3a07df7705c04c0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:17.041933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac6322-e1eb-4427-960e-a4f26fecbfdc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:17.060448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 913/1828] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:16 2022] PUT /identity/v3/projects/8647fb57a7704a438e6041d97462f797/users/47b5e0062da841f4b3a07df7705c04c0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:17.078747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:17.078747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:17.078747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:17.127634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] MFA Rules not processed for user `47b5e0062da841f4b3a07df7705c04c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:17.129291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:17.155296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 916/1829] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:16 2022] PUT /identity/v3/projects/bfb5cccc3e88497a96eb3c780e4d04fa/users/fa658a8b2ac94bc2b6fe1c8366657c8a/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:17.173293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:17.175567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:17.175567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:17.247698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] MFA Rules not processed for user `fa658a8b2ac94bc2b6fe1c8366657c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:17.251140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:17.331364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 914/1830] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:17.473734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 917/1831] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:17.479099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:17.731950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde920e6-f1b6-4a63-86a6-61644053e202 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:17.819525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:17.820221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:17.820515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:17.821093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:17.821837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:17.827607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.122395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 915/1832] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:17 2022] POST /identity/v3/projects => generated 383 bytes in 644 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:18.126516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.146153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae505cff-1cb9-45c2-94cb-ce02e2debdb5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.273109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.277039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.277039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.277039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.277039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.284539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.296382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.299323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:18.373311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f1728-21fb-4ffc-85b2-efdccb556294 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.379090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.382799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.382799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.382799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.458724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 918/1833] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:18 2022] POST /identity/v3/users => generated 298 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:18.460009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.460875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.461644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.474308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 916/1834] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:18.475999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6eb132-0083-4917-b01c-5c270ac00cff admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.484571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.485411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.485823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.486213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.487129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.488877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.499862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 919/1835] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:18.504210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.515000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d2d2c-e063-48dc-82e0-29ae80efb558 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.519670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.520562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.520854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.521140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] PATH_INFO: `/v3/projects/8b0280017ad645f889f9f6277f8ad770/users/1671156eab0543289622c22eaed5725a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.536529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0280017ad645f889f9f6277f8ad770, user_id=1671156eab0543289622c22eaed5725a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.539274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.593352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:18.659239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 917/1836] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:18 2022] PUT /identity/v3/projects/8b0280017ad645f889f9f6277f8ad770/users/1671156eab0543289622c22eaed5725a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:18.662114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.673992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba4c35-3c3d-4d76-ab01-d7fa835ddb52 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.746506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.747278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.747638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.747988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.749547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.751511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.763220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 920/1837] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:18.766995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.782093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00f67-174e-449c-ae80-99a8eeb6e279 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.794340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:18.809879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.810794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.811260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.811541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] PATH_INFO: `/v3/projects/8b0280017ad645f889f9f6277f8ad770/users/1671156eab0543289622c22eaed5725a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.813730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0280017ad645f889f9f6277f8ad770, user_id=1671156eab0543289622c22eaed5725a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:18.816458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c53f6-3af4-4866-8825-986e9a2839d4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:18.831751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 918/1838] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:18 2022] PUT /identity/v3/projects/8b0280017ad645f889f9f6277f8ad770/users/1671156eab0543289622c22eaed5725a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:18.844484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.844951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.845275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.871884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2553e-5db1-4128-a8a5-d485648d992d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:18.890236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] MFA Rules not processed for user `1671156eab0543289622c22eaed5725a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:18.892129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:18.950138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:18.950138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:18.950138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:18.950138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:18.999717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 919/1839] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:19.021044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:19.032509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:19.032509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 921/1840] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:19.596861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:19.598538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:19.612714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783362b-bfd3-42dc-96ba-14ce7b8b659a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:19.616269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:19.616912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:19.617197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:19.617478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:19.627895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:19.629431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:19.638863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 920/1841] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:19.724617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:19.864520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:19.873798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fb3-cf56-4dfd-9d7f-833f3ca13663 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:19.877582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:19.878286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:19.878575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:19.878948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:19.935244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:19.936683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c193f-0668-429d-897f-c72d7e58a86e service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:19.945532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 922/1842] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:20.524452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:20.527568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:20.592600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2a77-5f69-444d-981a-d743a9c20598 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:20.597380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:20.598282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:20.598695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:20.598974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] PATH_INFO: `/v3/users/fa658a8b2ac94bc2b6fe1c8366657c8a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:20.600203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa658a8b2ac94bc2b6fe1c8366657c8a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:20.602206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:20.801603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa658a8b2ac94bc2b6fe1c8366657c8a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:20.802466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa658a8b2ac94bc2b6fe1c8366657c8a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:20.900921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa658a8b2ac94bc2b6fe1c8366657c8a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:20.901611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa658a8b2ac94bc2b6fe1c8366657c8a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:20.909747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa658a8b2ac94bc2b6fe1c8366657c8a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:20.930183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 921/1843] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:20 2022] DELETE /identity/v3/users/fa658a8b2ac94bc2b6fe1c8366657c8a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:20.935371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:20.935937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:20.936103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:20.956693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:20.957997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:21.025947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 923/1844] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:21.035649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:21.037183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:21.096484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b3291-7399-4689-b049-e0d34a1fc25c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:21.100169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:21.100830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:21.101117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:21.101405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:21.111981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:21.113424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:21.123974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 922/1845] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:21.754910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:21.756492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:21.770168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f7867-124e-47c6-bb52-bce2b8726760 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:21.780678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:21.782568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:21.782568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:21.782871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] PATH_INFO: `/v3/projects/bfb5cccc3e88497a96eb3c780e4d04fa` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:21.784089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb5cccc3e88497a96eb3c780e4d04fa)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:21.786351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:21.948242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb5cccc3e88497a96eb3c780e4d04fa', 'request_id': 'req-1ddc500b-a5eb-4441-8469-1199382eb512'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:21.954500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb5cccc3e88497a96eb3c780e4d04fa', 'request_id': 'req-1ddc500b-a5eb-4441-8469-1199382eb512'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:21.961612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb5cccc3e88497a96eb3c780e4d04fa', 'request_id': 'req-1ddc500b-a5eb-4441-8469-1199382eb512'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.082901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] The token cache is being invalidate because project bfb5cccc3e88497a96eb3c780e4d04fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:22.083393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.084326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.086149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 924/1846] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:21 2022] DELETE /identity/v3/projects/bfb5cccc3e88497a96eb3c780e4d04fa => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:22.090528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:22.105101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15b20d-99fa-421e-adea-c47963c415d1 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:22.163734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:22.164617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:22.164909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:22.165196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] PATH_INFO: `/v3/users/1671156eab0543289622c22eaed5725a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:22.166423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1671156eab0543289622c22eaed5725a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:22.168396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:22.401082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1671156eab0543289622c22eaed5725a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.401797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1671156eab0543289622c22eaed5725a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.479202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:22.546396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1671156eab0543289622c22eaed5725a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.547061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1671156eab0543289622c22eaed5725a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.554516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a213ac8e-b9c4-4b89-a621-ad096a55c915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1671156eab0543289622c22eaed5725a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:22.571987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 923/1847] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:22 2022] DELETE /identity/v3/users/1671156eab0543289622c22eaed5725a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:23.067678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:23.069729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:23.084924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc500b-a5eb-4441-8469-1199382eb512 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:23.170961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:23.171879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:23.172162 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:23.172440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] PATH_INFO: `/v3/projects/8b0280017ad645f889f9f6277f8ad770` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:23.173647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0280017ad645f889f9f6277f8ad770)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:23.175904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.053391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:24.217191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0280017ad645f889f9f6277f8ad770', 'request_id': 'req-44b2acb2-707f-4909-b760-90cbbb58e983'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:24.223470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0280017ad645f889f9f6277f8ad770', 'request_id': 'req-44b2acb2-707f-4909-b760-90cbbb58e983'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:24.229961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0280017ad645f889f9f6277f8ad770', 'request_id': 'req-44b2acb2-707f-4909-b760-90cbbb58e983'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:24.338505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] The token cache is being invalidate because project 8b0280017ad645f889f9f6277f8ad770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:24.338741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:24.339362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-44b2acb2-707f-4909-b760-90cbbb58e983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:24.340924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 925/1848] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:23 2022] DELETE /identity/v3/projects/8b0280017ad645f889f9f6277f8ad770 => generated 0 bytes in 1274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:24.346098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.346604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.346834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.365762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:24.366987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:24.428830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 924/1849] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:24.440465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.441119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.441489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.464705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:24.466113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:24.474818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 926/1850] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:24.479213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.489152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200888-3bb6-401e-9c1f-86ab75fe9802 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.497637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.497637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.497637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.497637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.498181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:24.498181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.498181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.510196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 925/1851] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:24.514503 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.523852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b8af3-8b9a-4284-8b4e-5a858214909b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.528523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.529168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.529453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.529733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.530462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.532402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.628571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 927/1852] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:24 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:24.632769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.643495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc894a-8ba2-4143-b049-6ea56f10b4e5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.699458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.700133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.700418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.700698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.701397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.703330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.828051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 926/1853] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:24 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:24.830220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.844552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc31438-827a-46ae-9dd4-62b795837194 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.849846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.850574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.850868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.851147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.852038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.854226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.866216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 928/1854] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:24.870527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.881062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f441-af07-4e2c-96c9-e2206d84f5d0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.885951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.886762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.887083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.887366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.888106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.889738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e56f-53b0-4ccd-856e-6f965d8f2f88 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:24.900700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 927/1855] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:24.904943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:24.914832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ae5b1-fed1-4c72-947c-c04650031457 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:24.919955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:24.920830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:24.921122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:24.921401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] PATH_INFO: `/v3/projects/e3bfadb86bcc40e38942a2ea1337e383/users/fb4b0abe878d41d392978555488b5dbb/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:24.938442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3bfadb86bcc40e38942a2ea1337e383, user_id=fb4b0abe878d41d392978555488b5dbb, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:24.941191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:25.153938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 929/1856] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:24 2022] PUT /identity/v3/projects/e3bfadb86bcc40e38942a2ea1337e383/users/fb4b0abe878d41d392978555488b5dbb/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:25.165545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:25.165875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:25.166970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:25.186502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:25.221162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] MFA Rules not processed for user `fb4b0abe878d41d392978555488b5dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:25.221515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:25.292710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 928/1857] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:25.304076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:25.306887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:25.367395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58e56-6b82-4e00-bc9a-1ee05d6018e6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:25.371639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:25.372300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:25.372587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:25.373160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:25.374305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:25.375934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:25.382404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:25.384014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd24fe-4f6c-4923-b574-fe5588ee55b6 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:25.393812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 930/1858] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:25.442675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263029-6c32-4dbe-862a-b01adac702b3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:25.447348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:25.448235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:25.448528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:25.448807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] PATH_INFO: `/v3/users/5977443511e246738c4467f93d8454d2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:25.450067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5977443511e246738c4467f93d8454d2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:25.452003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:25.684123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5977443511e246738c4467f93d8454d2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:25.684722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5977443511e246738c4467f93d8454d2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:25.769372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5977443511e246738c4467f93d8454d2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:25.770045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5977443511e246738c4467f93d8454d2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:25.777442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5977443511e246738c4467f93d8454d2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:25.795413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 929/1859] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:25 2022] DELETE /identity/v3/users/5977443511e246738c4467f93d8454d2 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:25.800033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:25.800421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:25.800968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:25.820768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:25.822032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:25.879959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 931/1860] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:25.892688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:25.894266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:25.975883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad4cf-7592-4f98-be18-78699e46502a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:25.983769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:25.984535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:25.996204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:25.996204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:25.997399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:25.999088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:26.012544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 930/1861] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:26.238626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:26.939724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:27.425540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:27.427907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:27.448830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc1fd1-6135-4df7-be66-570a464d1d31 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:27.460103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:27.461554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:27.462165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:27.464301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] PATH_INFO: `/v3/projects/08aa621fa9914449a1a51066e25673eb` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:27.466645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08aa621fa9914449a1a51066e25673eb)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:27.469948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:27.633477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08aa621fa9914449a1a51066e25673eb', 'request_id': 'req-b4825ff7-7536-4d74-9a64-6e5519e1224f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:27.648429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08aa621fa9914449a1a51066e25673eb', 'request_id': 'req-b4825ff7-7536-4d74-9a64-6e5519e1224f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:27.664233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08aa621fa9914449a1a51066e25673eb', 'request_id': 'req-b4825ff7-7536-4d74-9a64-6e5519e1224f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:27.773056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] The token cache is being invalidate because project 08aa621fa9914449a1a51066e25673eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:27.773861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:27.774756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:27.776984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 932/1862] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:27 2022] DELETE /identity/v3/projects/08aa621fa9914449a1a51066e25673eb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:27.785270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:27.796272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10682fb-ae93-435c-9410-d139ddfd9196 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:27.884084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:27.884646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:27.884933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:27.885217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] PATH_INFO: `/v3/users/47b5e0062da841f4b3a07df7705c04c0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:27.886548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b5e0062da841f4b3a07df7705c04c0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:27.889803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:28.156659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b5e0062da841f4b3a07df7705c04c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:28.157796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b5e0062da841f4b3a07df7705c04c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:28.263048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:28.263727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b5e0062da841f4b3a07df7705c04c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:28.264452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b5e0062da841f4b3a07df7705c04c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:28.272084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-810c4489-46ab-4e4a-a27c-6c0b432bbb5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b5e0062da841f4b3a07df7705c04c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:28.289185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 931/1863] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:27 2022] DELETE /identity/v3/users/47b5e0062da841f4b3a07df7705c04c0 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:28.787142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:28.789312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:28.819847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825ff7-7536-4d74-9a64-6e5519e1224f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:28.931137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:28.932713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:28.933200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:28.933459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] PATH_INFO: `/v3/projects/8647fb57a7704a438e6041d97462f797` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:28.934780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8647fb57a7704a438e6041d97462f797)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:28.936976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.091069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:29.110752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8647fb57a7704a438e6041d97462f797', 'request_id': 'req-8827b931-7b0b-4b14-807f-bcfc52081dc8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:29.116764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8647fb57a7704a438e6041d97462f797', 'request_id': 'req-8827b931-7b0b-4b14-807f-bcfc52081dc8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:29.122086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8647fb57a7704a438e6041d97462f797', 'request_id': 'req-8827b931-7b0b-4b14-807f-bcfc52081dc8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:29.292661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] The token cache is being invalidate because project 8647fb57a7704a438e6041d97462f797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:29.293121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:29.293992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8827b931-7b0b-4b14-807f-bcfc52081dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:29.299647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 933/1864] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:28 2022] DELETE /identity/v3/projects/8647fb57a7704a438e6041d97462f797 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:29.301834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.302325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.302575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.322730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:29.323956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:29.386057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 932/1865] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:29.395048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.395048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.395718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.416348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:29.416348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:29.424788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 934/1866] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:29.429215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.439956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23e031-f6cd-49de-9658-d61724342296 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.447592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.447592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.447592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.447592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.447932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:29.447932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.448799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.460980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 933/1867] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:29.465556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.475119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb316f-7812-42e3-ab2e-3f149a60e38a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.480444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.481369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.481653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.482007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.482763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.484736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.594033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 935/1868] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:29 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:29.598289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.609191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa9b70-ffef-4c86-a43b-186efaf0c39e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.664257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.664929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.665222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.666023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.666740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.668677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.832719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 934/1869] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:29 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:29.837122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.847939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6b23d-3533-4e47-9c0b-6247a224291e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.852577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.853978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.854262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.854536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.855273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.857219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.872736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 936/1870] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:29.874059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.885518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201fede-7763-44b3-ab96-b2b9a9a13d69 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.891815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.892503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.893125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.893478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.894870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.897876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f04ba2-9654-41ad-be02-46dbe1bdb8af admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:29.910155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 935/1871] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:29.914380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:29.926536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe569faa-a861-4532-8dde-fb7df603784c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:29.932332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:29.933084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:29.933369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:29.933854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] PATH_INFO: `/v3/projects/b0dc116684ad4fee8b3ba0a83acaf855/users/8ea9420a857d401db090e89a6bf6a49f/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:29.951260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0dc116684ad4fee8b3ba0a83acaf855, user_id=8ea9420a857d401db090e89a6bf6a49f, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:29.954163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.050054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 937/1872] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:29 2022] PUT /identity/v3/projects/b0dc116684ad4fee8b3ba0a83acaf855/users/8ea9420a857d401db090e89a6bf6a49f/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:30.060409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.060708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.060993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.101524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] MFA Rules not processed for user `8ea9420a857d401db090e89a6bf6a49f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:30.103070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:30.175444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 936/1873] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:30.179938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.243325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42523c5-cb28-4ad0-93e7-605dff111e63 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.248033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.248691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.248977 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.249255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.250271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.252081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.290184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:30.353941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 938/1874] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:30 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:30.358009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.368696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7c76e-5910-42a7-9a3d-965b1770851b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.424385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.424947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.425239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.425523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.426320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.428191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.525452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 937/1875] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:30 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:30.525623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.535583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d6cd-c502-46fa-851c-ec553216035d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.540350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.541038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.541320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.541599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.542388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.543982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.554956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 939/1876] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:30.559482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.570447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d70c-21f3-44f1-9491-2efddbbcd3cf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.575716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.576381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.576672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.577000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.577688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.579335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59619941-5eef-452e-ae4d-1690b19bf75f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.589280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 938/1877] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:30.593546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.603700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1f598-1ce6-44c3-99e2-3ba509eb3781 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.608576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.609461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.609752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.610062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] PATH_INFO: `/v3/projects/93f4664de3684d95aba50b83bb554089/users/ce66cf8a13b64d1ab41e881db0558856/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.624994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f4664de3684d95aba50b83bb554089, user_id=ce66cf8a13b64d1ab41e881db0558856, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.627688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:30.750939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 940/1878] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:30 2022] PUT /identity/v3/projects/93f4664de3684d95aba50b83bb554089/users/ce66cf8a13b64d1ab41e881db0558856/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:30.762849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.763145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.763427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.796201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] MFA Rules not processed for user `ce66cf8a13b64d1ab41e881db0558856`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:30.797515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:30.863568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 939/1879] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:30.869644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:30.938733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde274d-3acc-4dba-be64-68e0c5800ee5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:30.943435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:30.944100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:30.944392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:30.944712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:30.945417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:30.947510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.099123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 941/1880] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:30 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:31.104525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.120850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b78ed-269b-4925-affc-eca8ca501d96 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.204281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.204743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.205035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.205331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.206226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.208308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.358247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 940/1881] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:31 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:31.364692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.375139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9faf4-7a98-4991-888c-25c84a60b7b7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.379843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.380580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.380946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.381291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.382159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.383843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.395750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 942/1882] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:31.405056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.415337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f398-112e-407b-b1d9-e7c72bd660cc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.420783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.421717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.422040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.422324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] PATH_INFO: `/v3/projects/8584ea08db4241668619a8a0bfb4857f/users/bdc2d6db110e4b58b0101914d8603899/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.437660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8584ea08db4241668619a8a0bfb4857f, user_id=bdc2d6db110e4b58b0101914d8603899, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.440379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.627397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 941/1883] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:31 2022] PUT /identity/v3/projects/8584ea08db4241668619a8a0bfb4857f/users/bdc2d6db110e4b58b0101914d8603899/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:31.631488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.641895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e639-1ad1-435e-8eac-a2c61d9f1823 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.698557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.699245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.699723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.699868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.700610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.702295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b57a36-6080-41da-b833-59d46360e9cf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.712717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 943/1884] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:31.720642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.731360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea483533-2a16-49c0-9ddf-e6a6da70ff2c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.736550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.737475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.737794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.738083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] PATH_INFO: `/v3/projects/8584ea08db4241668619a8a0bfb4857f/users/bdc2d6db110e4b58b0101914d8603899/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.740266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8584ea08db4241668619a8a0bfb4857f, user_id=bdc2d6db110e4b58b0101914d8603899, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.743048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.757793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 942/1885] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:31 2022] PUT /identity/v3/projects/8584ea08db4241668619a8a0bfb4857f/users/bdc2d6db110e4b58b0101914d8603899/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:31.768806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.769079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.769364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.796274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] MFA Rules not processed for user `bdc2d6db110e4b58b0101914d8603899`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:31.797266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:31.866857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 944/1886] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:31.878199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:31.879969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:31.937075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503f414-6965-46b4-a78b-057bff5a8359 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:31.940929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:31.941636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:31.941961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:31.942284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:31.947548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:31.952937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:31.954502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e49e15-54e4-46dc-bfbf-29797e3e7e58 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:31.965253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 943/1887] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:33.138401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:33.139920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.217721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cbe69-a507-4899-986f-4f0dfdef950b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:33.223253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:33.223253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.223566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.224060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] PATH_INFO: `/v3/users/c1e7f6385e08458f9026a319e07bd8e1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.225358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e7f6385e08458f9026a319e07bd8e1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:33.227338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:33.238822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:33.491014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e7f6385e08458f9026a319e07bd8e1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:33.500002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb5b0-54aa-455c-a3dc-06c266a0ce54 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.500979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb5b0-54aa-455c-a3dc-06c266a0ce54 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.501446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb5b0-54aa-455c-a3dc-06c266a0ce54 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.503797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 944/1888] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:31:33.506822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e7f6385e08458f9026a319e07bd8e1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:33.519015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.519684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.520134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.541633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:33.552683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.604732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e7f6385e08458f9026a319e07bd8e1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:33.608773 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e7f6385e08458f9026a319e07bd8e1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:33.610877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e7f6385e08458f9026a319e07bd8e1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:33.658778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 945/1889] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:33 2022] DELETE /identity/v3/users/c1e7f6385e08458f9026a319e07bd8e1 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:33.662029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:33.666975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.698085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 945/1890] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:33.701052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:33.701052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.715796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c213f-46b5-4c34-b546-d1c68c3c3138 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:33.785851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:33.786620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.786987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.787346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.789751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.799527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e25e1-4884-4312-a98c-ef1c27a188d0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:33.816784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:33.816784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.816784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.816784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] PATH_INFO: `/v3/users/01c91682c913412392874ea398100c7c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.818452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c91682c913412392874ea398100c7c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:33.822788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:33.914403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:33.916412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7605f03-b1e1-4f7f-a9fd-35fbc44b352e service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:33.927800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 946/1891] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:33.929448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:33.929750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:33.930075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:33.959362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:33.960764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:33.991136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 947/1892] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:33.992309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:33.995532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:34.116714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca4aa5-a06a-47d4-b957-eb735893a76a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:34.134340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:34.134647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:34.134939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:34.150008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:34.150275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c91682c913412392874ea398100c7c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:34.153733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:34.155395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:34.157076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c91682c913412392874ea398100c7c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:34.179457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 948/1893] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:34.180567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:34.182829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:34.252292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d35e2-3eeb-4679-b11d-d61788fc01ed service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:34.256055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:34.256713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:34.257003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:34.257292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:34.266765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:34.268192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0989439-54d1-4f8f-baaf-4df726d110ba service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:34.278550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 949/1894] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:34.354137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c91682c913412392874ea398100c7c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:34.355681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c91682c913412392874ea398100c7c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:34.357355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c91682c913412392874ea398100c7c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:34.370940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 946/1895] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:33 2022] DELETE /identity/v3/users/01c91682c913412392874ea398100c7c => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:34.378511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:34.378511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:34.378511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:34.394384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:34.395693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:34.455182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 950/1896] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:34.466374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:34.467819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:34.526124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfa68-c051-429f-a3eb-7c46688062e9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:34.529777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:34.530521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:34.530814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:34.531097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:34.541354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:34.542860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:34.554100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 947/1897] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:34.882037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:34.884784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:34.896898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042da84d-60ce-4447-a48b-03569bb89fef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:34.902680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:34.903627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:34.903912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:34.904192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] PATH_INFO: `/v3/projects/b366acdadbf840488244d6c6d4e3b2d6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:34.905459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b366acdadbf840488244d6c6d4e3b2d6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:34.910554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:35.149788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b366acdadbf840488244d6c6d4e3b2d6', 'request_id': 'req-960f8b10-8b89-43c2-817d-3aa8acc4a20f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.164758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b366acdadbf840488244d6c6d4e3b2d6', 'request_id': 'req-960f8b10-8b89-43c2-817d-3aa8acc4a20f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.179519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:35.182266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:35.202886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef21f-ad3e-4024-956d-92c22e87ae5a service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:35.309532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:35.310523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:35.310919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:35.311393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] PATH_INFO: `/v3/projects/63a7e5d3bb8545f1b6c3b92ba4889cad` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:35.313567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a7e5d3bb8545f1b6c3b92ba4889cad)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:35.320416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:35.382486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b366acdadbf840488244d6c6d4e3b2d6', 'request_id': 'req-960f8b10-8b89-43c2-817d-3aa8acc4a20f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.400382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] The token cache is being invalidate because project b366acdadbf840488244d6c6d4e3b2d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:35.400740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.401677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.409712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 951/1898] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:34 2022] DELETE /identity/v3/projects/b366acdadbf840488244d6c6d4e3b2d6 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:35.410871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:35.412689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:35.416779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:35.436742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8b10-8b89-43c2-817d-3aa8acc4a20f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:35.441634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:35.442609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:35.442896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:35.443183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] PATH_INFO: `/v3/users/3d61dd1583e24979b3627c9968381ce4` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:35.444447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d61dd1583e24979b3627c9968381ce4)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:35.446604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:35.499930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a7e5d3bb8545f1b6c3b92ba4889cad', 'request_id': 'req-15a234c7-288d-4ce8-9356-1c58db838bad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.507167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a7e5d3bb8545f1b6c3b92ba4889cad', 'request_id': 'req-15a234c7-288d-4ce8-9356-1c58db838bad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.512943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a7e5d3bb8545f1b6c3b92ba4889cad', 'request_id': 'req-15a234c7-288d-4ce8-9356-1c58db838bad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.655684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:35.660690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] The token cache is being invalidate because project 63a7e5d3bb8545f1b6c3b92ba4889cad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:35.661039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.665155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.665155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 948/1899] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:35 2022] DELETE /identity/v3/projects/63a7e5d3bb8545f1b6c3b92ba4889cad => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:35.668353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:35.669747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:35.748972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a234c7-288d-4ce8-9356-1c58db838bad admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:35.753761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:35.754672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:35.754959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:35.755235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] PATH_INFO: `/v3/users/4e95b5e5e2e14b47b31dd6002fc1b437` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:35.756501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e95b5e5e2e14b47b31dd6002fc1b437)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:35.758646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:35.779015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d61dd1583e24979b3627c9968381ce4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.780003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d61dd1583e24979b3627c9968381ce4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.936367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d61dd1583e24979b3627c9968381ce4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.937110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d61dd1583e24979b3627c9968381ce4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.945265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d61dd1583e24979b3627c9968381ce4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:35.965676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 952/1900] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:35 2022] DELETE /identity/v3/users/3d61dd1583e24979b3627c9968381ce4 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:36.080560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e95b5e5e2e14b47b31dd6002fc1b437'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:36.093152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e95b5e5e2e14b47b31dd6002fc1b437'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:36.204450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e95b5e5e2e14b47b31dd6002fc1b437'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:36.205942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e95b5e5e2e14b47b31dd6002fc1b437'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:36.206558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e95b5e5e2e14b47b31dd6002fc1b437'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:36.219302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 949/1901] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:35 2022] DELETE /identity/v3/users/4e95b5e5e2e14b47b31dd6002fc1b437 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:36.571127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:36.572970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:36.686542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3aaf-fdc5-41d5-a54f-0e5bda62de35 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:36.773145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:36.774644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:36.775635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:36.776062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] PATH_INFO: `/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:36.778557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5d8fc6b83a470b911fddbd798b6e7f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:36.781002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:36.936801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:36.938846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:36.953536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b680b-f2e6-46a5-81e5-ad551c88d1db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:36.959706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:36.960583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:36.960872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:36.961150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] PATH_INFO: `/v3/projects/4a26d10b918c4d65bfc06075f630a9a2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:36.963243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a26d10b918c4d65bfc06075f630a9a2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:36.965893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.028832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5d8fc6b83a470b911fddbd798b6e7f', 'request_id': 'req-03007db6-e147-4b10-a674-1415035d3fbd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.035206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5d8fc6b83a470b911fddbd798b6e7f', 'request_id': 'req-03007db6-e147-4b10-a674-1415035d3fbd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.160197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a26d10b918c4d65bfc06075f630a9a2', 'request_id': 'req-62d93e99-c868-4f5a-8b59-25b422bfdaf4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.166325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a26d10b918c4d65bfc06075f630a9a2', 'request_id': 'req-62d93e99-c868-4f5a-8b59-25b422bfdaf4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.171748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a26d10b918c4d65bfc06075f630a9a2', 'request_id': 'req-62d93e99-c868-4f5a-8b59-25b422bfdaf4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.179452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5d8fc6b83a470b911fddbd798b6e7f', 'request_id': 'req-03007db6-e147-4b10-a674-1415035d3fbd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.191707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] The token cache is being invalidate because project ae5d8fc6b83a470b911fddbd798b6e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:37.192004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.192656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-03007db6-e147-4b10-a674-1415035d3fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.194788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 953/1902] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:36 2022] DELETE /identity/v3/projects/ae5d8fc6b83a470b911fddbd798b6e7f => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:37.200486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf02b05-afda-4a2f-8f64-cd1a8d0c7b5c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.200761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf02b05-afda-4a2f-8f64-cd1a8d0c7b5c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.201047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf02b05-afda-4a2f-8f64-cd1a8d0c7b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.220330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5bf02b05-afda-4a2f-8f64-cd1a8d0c7b5c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:37.221554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5bf02b05-afda-4a2f-8f64-cd1a8d0c7b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:37.291187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 954/1903] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:37.300760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.301067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.301337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.325585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:37.326972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:37.327827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] The token cache is being invalidate because project 4a26d10b918c4d65bfc06075f630a9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:37.332887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.332887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62d93e99-c868-4f5a-8b59-25b422bfdaf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:37.332887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 950/1904] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:36 2022] DELETE /identity/v3/projects/4a26d10b918c4d65bfc06075f630a9a2 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:37.336622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50226bd-513c-428a-af47-bcae230910a0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.337160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50226bd-513c-428a-af47-bcae230910a0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.337646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50226bd-513c-428a-af47-bcae230910a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.344972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 955/1905] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:37.349965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.361921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a54e55-3183-4a13-9876-cf33c50022b4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.365595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f50226bd-513c-428a-af47-bcae230910a0 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:37.367625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f50226bd-513c-428a-af47-bcae230910a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:37.367972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.369285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.369805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.369924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.384772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:37.384772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.384772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.385063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 951/1906] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:37.389811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 956/1907] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:37.391382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.391531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.391669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.392113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.402001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f097c8-be1a-46c8-9d87-a5237ecbb09a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.406977 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.407641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.407928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.408215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.408908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.410985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.412482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:37.413730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:37.422411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 952/1908] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:37.426772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.436264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd8e3a-bf71-46fd-857d-c9cdfd1b4c0b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.441000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.441648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.441975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.442285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.443035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:37.443347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.444942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.457354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 953/1909] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:31:37.461638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.472578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767de8b-54e3-4512-bbd4-bd515dabae39 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.477247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.477957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.478229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.478511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.479210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.481372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.603550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 957/1910] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:37.604452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.604670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 954/1911] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/projects => generated 391 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:37.609458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.679159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f9ca7-99a2-4c4c-938d-e1b9f5332db7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.703492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38bf7b-ad5f-4dd1-a251-3c8421a9509a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.710116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.711037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.711325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.711842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.712591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.714561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.752797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.753489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.753799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.754094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.754803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.756743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.864853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 958/1912] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:37.874353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 955/1913] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:37 2022] POST /identity/v3/users => generated 296 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:37.874457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.875206 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.887484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8e19-3ac1-4012-88a9-be11bc2d5b7b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.887703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b169d1a-7cb2-4159-925a-5fdede649f71 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.892362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.893542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.893697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.894011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.894347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.894724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.895117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.895228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.895601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.896530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.896741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.899458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.911152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 956/1914] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:37.913904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 959/1915] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:37.925871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.933397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.940240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfd6a0-76fc-49d0-8633-a0c01d58c61d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.945914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0d996-401a-4047-ae2b-1754ec9e3b7b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.948436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.949401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.949966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.950850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.951742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.952025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.952692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.953145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.953423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.954344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:37.955730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.955954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:37.974713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 960/1916] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:37.974870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 957/1917] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:37.974964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.996473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac46b6-8de4-4eb9-8078-fe7effd0621c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:37.996473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:37.996473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:37.996473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:37.996968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] PATH_INFO: `/v3/projects/b0ea3bb43e3649e0b9bcbc62244d91e1/users/45a2bac6401d45888b31aa7b09838ab1/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:37.997201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:37.998911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.014311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ea3bb43e3649e0b9bcbc62244d91e1, user_id=45a2bac6401d45888b31aa7b09838ab1, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:38.017390 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:38.019367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34947685-e650-493e-bcbe-1cb4ff8aecaf admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:38.032485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:38.033834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.034129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.035736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:38.036372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] PATH_INFO: `/v3/users/fb4b0abe878d41d392978555488b5dbb` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.038090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4b0abe878d41d392978555488b5dbb)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:38.044326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:38.121211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 961/1918] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:37 2022] PUT /identity/v3/projects/b0ea3bb43e3649e0b9bcbc62244d91e1/users/45a2bac6401d45888b31aa7b09838ab1/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:38.122197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:38.203092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f6405-5cc1-4e8e-a5c6-097153366181 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:38.208532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:38.209653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.209993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.210302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] PATH_INFO: `/v3/projects/f5b345e9c81547e2bdc60edc33bcb157/users/1ecf31b1806942c89d09677f0d31d73b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.225641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b345e9c81547e2bdc60edc33bcb157, user_id=1ecf31b1806942c89d09677f0d31d73b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:38.229004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453ce8e-b401-4106-8b42-c591fa1ca573 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:38.293163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4b0abe878d41d392978555488b5dbb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:38.308775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4b0abe878d41d392978555488b5dbb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:38.379342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 962/1919] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:38 2022] PUT /identity/v3/projects/f5b345e9c81547e2bdc60edc33bcb157/users/1ecf31b1806942c89d09677f0d31d73b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:38.381526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3db5e-86ff-4702-bee1-062af92b9269 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.382059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3db5e-86ff-4702-bee1-062af92b9269 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.382259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3db5e-86ff-4702-bee1-062af92b9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.413056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4b0abe878d41d392978555488b5dbb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:38.414309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4b0abe878d41d392978555488b5dbb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:38.414936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-024bbdc0-5b9f-4e6b-a7a1-b6ed1e669242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4b0abe878d41d392978555488b5dbb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:38.420446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-74c3db5e-86ff-4702-bee1-062af92b9269 None None] MFA Rules not processed for user `45a2bac6401d45888b31aa7b09838ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:38.421715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-74c3db5e-86ff-4702-bee1-062af92b9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.438703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 958/1920] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:37 2022] DELETE /identity/v3/users/fb4b0abe878d41d392978555488b5dbb => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:38.440395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.440884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.441253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.473652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] MFA Rules not processed for user `1ecf31b1806942c89d09677f0d31d73b`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:38.475236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.505311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 963/1921] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:38.505311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.505311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.505311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.527237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:38.528612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.546127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 959/1922] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:38.550210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:38.557492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.632166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 964/1923] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:38.633725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:38.635312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.647566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e3610-c383-40ff-948f-e5f65cd4cf9e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:38.653178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:38.653932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.654223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.654510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.655222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:38.657647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:38.685519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571e075-69e6-43cc-9a84-3a65b7352ef8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:38.691578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:38.692342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.692539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.692829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.693585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:38.696842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:38.845917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 960/1924] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:38 2022] POST /identity/v3/projects => generated 393 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:38.847089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:38.850867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.851919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 965/1925] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:38 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:38.852908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:38.855660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:38.922874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baef58f-4e39-4c16-bb8e-42e760100446 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:38.926688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:38.927360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:38.927656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:38.927939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:38.930493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fdcc9-dac7-4a3a-a908-025d1e33a751 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.022642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.023730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.023963 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.024259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] PATH_INFO: `/v3/users/6de72a8e5c494b318dad785f397fe5ba` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.026360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de72a8e5c494b318dad785f397fe5ba)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.028314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.041350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.042925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.061990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 966/1926] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:39.062780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.084844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd58bbc-d0b4-47e6-a614-426983aa9306 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.092662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.092867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.093148 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.095575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.095575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.097554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.298742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 967/1927] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:39 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:39.299577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.309801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742700bd-6ad2-47ee-accc-753c728b8200 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.315223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.315900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.316193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.316498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.317348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.319454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.368978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de72a8e5c494b318dad785f397fe5ba'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:39.393223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de72a8e5c494b318dad785f397fe5ba'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:39.458565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 968/1928] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:39 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:39.463400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.478406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df3a00-3bf6-4697-a4c1-06bcbd56529d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.483880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de72a8e5c494b318dad785f397fe5ba'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:39.484799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de72a8e5c494b318dad785f397fe5ba'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:39.485625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de72a8e5c494b318dad785f397fe5ba'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:39.499853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.500663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.500963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.501250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.503455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 961/1929] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:38 2022] DELETE /identity/v3/users/6de72a8e5c494b318dad785f397fe5ba => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:39.504336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.505603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.505735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:39.507293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20daeb-22e5-49eb-ab1f-b367683200fc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.528642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 969/1930] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:39.530166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.530469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.531427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.549524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:39.551145 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:39.578156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf34a3-91ae-4bf0-9a05-d4a77acb22e0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.585550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.586258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.586562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.593592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.593592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.593592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.600057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 962/1931] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:39.601043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.603135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:39.616481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c12bc-2e51-47bc-9812-96bd95c11282 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.626580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.627514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.627797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.628084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] PATH_INFO: `/v3/projects/955f78b1798141f5b068e66cbf426145/users/1b815d109a2946388e402b2ea7066fde/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.638534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 970/1932] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:39.639414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.649243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cefcb-eefc-4a4f-9cb7-fe6c9e21450e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.650930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=955f78b1798141f5b068e66cbf426145, user_id=1b815d109a2946388e402b2ea7066fde, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.653887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.654001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.655189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.655868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.656229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] PATH_INFO: `/v3/projects/6ccbb85fd6e54c749579e246230cbbb5/users/c32f2277e4914d95b70fd9810d330361/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.658429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:39.672728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccbb85fd6e54c749579e246230cbbb5, user_id=c32f2277e4914d95b70fd9810d330361, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.675476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.787489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 963/1933] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:39 2022] PUT /identity/v3/projects/955f78b1798141f5b068e66cbf426145/users/1b815d109a2946388e402b2ea7066fde/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:39.787489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.788823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:39.792574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 971/1934] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:39 2022] PUT /identity/v3/projects/6ccbb85fd6e54c749579e246230cbbb5/users/c32f2277e4914d95b70fd9810d330361/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:39.794244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:39.796899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:39.873634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e250-c7e9-4ec8-a5e3-a484f1f23cda admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.933674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4936-adfd-4137-a56a-6f052c2645b3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:39.943104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.943989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.944271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.944731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] PATH_INFO: `/v3/projects/e3bfadb86bcc40e38942a2ea1337e383` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:39.946472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3bfadb86bcc40e38942a2ea1337e383)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:39.948994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:39.994770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:39.995445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:39.995743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:39.996023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.016980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.018511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.030702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 964/1935] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:40.031580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.041604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd3bd7-53c7-47e4-9ee0-5a765cc20852 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.046646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.048754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.048754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.048754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.048754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.050249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.061174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 965/1936] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:40.062189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.075093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224a493-e678-461a-9125-f3e7dff4ef09 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.080090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.080785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.081202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.081493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.082261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.083899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.095524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 966/1937] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:40.096441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.107310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793de2ff-7f34-4e53-9b39-6c6d37ff83f1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.114311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.115519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.115989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.116717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] PATH_INFO: `/v3/projects/955f78b1798141f5b068e66cbf426145/users/1b815d109a2946388e402b2ea7066fde/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.119469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955f78b1798141f5b068e66cbf426145, user_id=1b815d109a2946388e402b2ea7066fde, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.122508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.135135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 967/1938] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:40 2022] PUT /identity/v3/projects/955f78b1798141f5b068e66cbf426145/users/1b815d109a2946388e402b2ea7066fde/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:40.135857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.205042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2924-8123-438c-b959-3badc01d6423 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.210136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.211075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.211400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.211682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] PATH_INFO: `/v3/projects/6ccbb85fd6e54c749579e246230cbbb5/users/c32f2277e4914d95b70fd9810d330361/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.214576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccbb85fd6e54c749579e246230cbbb5, user_id=c32f2277e4914d95b70fd9810d330361, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.217471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38acf3-ab7e-4ca3-bc39-c595281768db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.228398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 968/1939] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:40 2022] PUT /identity/v3/projects/6ccbb85fd6e54c749579e246230cbbb5/users/c32f2277e4914d95b70fd9810d330361/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:40.237367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd68e-5ed6-4649-8ece-91f63c6ffeaa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.237367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd68e-5ed6-4649-8ece-91f63c6ffeaa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.237367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd68e-5ed6-4649-8ece-91f63c6ffeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.258228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-faecd68e-5ed6-4649-8ece-91f63c6ffeaa None None] MFA Rules not processed for user `1b815d109a2946388e402b2ea7066fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:40.259991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-faecd68e-5ed6-4649-8ece-91f63c6ffeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.292538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3bfadb86bcc40e38942a2ea1337e383', 'request_id': 'req-af84ca97-2072-452e-a892-71649a77d691'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.299105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3bfadb86bcc40e38942a2ea1337e383', 'request_id': 'req-af84ca97-2072-452e-a892-71649a77d691'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.344003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 969/1940] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:40.345649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.346035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.361890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.373411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] MFA Rules not processed for user `c32f2277e4914d95b70fd9810d330361`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:40.374696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.426629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3bfadb86bcc40e38942a2ea1337e383', 'request_id': 'req-af84ca97-2072-452e-a892-71649a77d691'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.439664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] The token cache is being invalidate because project e3bfadb86bcc40e38942a2ea1337e383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:40.439973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.440617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.445363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 972/1941] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:39 2022] DELETE /identity/v3/projects/e3bfadb86bcc40e38942a2ea1337e383 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:40.445363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.447613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.473819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 970/1942] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:40.478078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.481401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.507651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84ca97-2072-452e-a892-71649a77d691 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.512342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.513162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.513276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.513546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.515880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.544142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d9552-1653-47fa-ac96-5e76ac7bd7a4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.549101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.549781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.550099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.550385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.556726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.599755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.601048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6eb53-6379-4f6c-b336-ca9bfbbeae72 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.612095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 973/1943] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:40.626437 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.626728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.627008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.639273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.642306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b657-0471-4bef-bc03-47d09b905108 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.645666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:40.646899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.656262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 971/1944] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:40.711382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 974/1945] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:40.719276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.719822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.719936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.741842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:40.743087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.745929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.759180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80722056-6635-4cf2-b33c-cc5ab05d21f9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.761230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.762422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.762717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.763351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.764611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:40.768656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 972/1946] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:40.771449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.773089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:40.781450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.782934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.785681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bcd57-5af1-4a58-9473-414e12980007 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.790227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.791563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.793481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.793481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] PATH_INFO: `/v3/projects/5f01694d5df04a6a86bcec0c94abc902` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.793481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f01694d5df04a6a86bcec0c94abc902)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.794340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 975/1947] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:40.795114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.799168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.805533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde844-25f2-4be1-8982-c26e283b3db6 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.811463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.812819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.812819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.812999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.813719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:40.814061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.815691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.829969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 976/1948] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:40.834361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.844578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d88e-5a67-4b8a-874f-4c0d81e68551 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:40.849730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:40.850459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:40.856398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:40.856398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:40.856398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:40.856398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:40.945227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f01694d5df04a6a86bcec0c94abc902', 'request_id': 'req-2de236c6-2f4f-4080-9ab0-b5da78727c2f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.951410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f01694d5df04a6a86bcec0c94abc902', 'request_id': 'req-2de236c6-2f4f-4080-9ab0-b5da78727c2f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.957109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f01694d5df04a6a86bcec0c94abc902', 'request_id': 'req-2de236c6-2f4f-4080-9ab0-b5da78727c2f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:40.983350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 977/1949] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:40 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:40.984548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:40.987368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.054273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05866d8-21e5-4e6d-af18-4869fdd2e27e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.060061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.060961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.061250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.061531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] PATH_INFO: `/v3/users/1ecf31b1806942c89d09677f0d31d73b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.063160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ecf31b1806942c89d09677f0d31d73b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.065466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.105736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] The token cache is being invalidate because project 5f01694d5df04a6a86bcec0c94abc902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:41.106580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.107335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.109680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 973/1950] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:40 2022] DELETE /identity/v3/projects/5f01694d5df04a6a86bcec0c94abc902 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:41.113986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.115379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.181301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de236c6-2f4f-4080-9ab0-b5da78727c2f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.239664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.240321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.240610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.240890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.242763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.301281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.309804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.329338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 974/1951] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:41.342534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.344321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.358852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e5f1-0961-4d5c-98a4-01c0842ef63f service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.366763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.366763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.366763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.366763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.367134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.368071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.478602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ecf31b1806942c89d09677f0d31d73b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.479333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ecf31b1806942c89d09677f0d31d73b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.533484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 975/1952] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:41 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:41.533484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.534021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.546779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0b591-4005-437f-8863-0493b662049e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.551639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.552506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.552792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.553070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] PATH_INFO: `/v3/users/52936cb1980e4728868f5614b5edefd0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.554421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52936cb1980e4728868f5614b5edefd0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.556419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.564232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ecf31b1806942c89d09677f0d31d73b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.564889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ecf31b1806942c89d09677f0d31d73b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.573735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ecf31b1806942c89d09677f0d31d73b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.591980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 978/1953] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:40 2022] DELETE /identity/v3/users/1ecf31b1806942c89d09677f0d31d73b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:41.592889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.594244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.656933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a8884-4058-4463-aceb-69829dced54e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.661625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.662324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.662606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.662888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.665843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.665843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86907445-e447-485a-ba7a-181d3be537c4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.675848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 979/1954] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:41.677081 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.677372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.677650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.697052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:41.698801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.709869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 980/1955] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:41.709869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.719846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093040-08b6-4f5a-9aaf-64bba4a5030a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.724421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.725138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.725421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.725700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.726461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.728903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.739930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 981/1956] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:41.740733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.742378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:41.798854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01edc8-7255-4b4f-8f29-d21b7e2eea0b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.802493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.803169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.803463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.803742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.813094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.814623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.824474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 982/1957] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:41.825444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:41.835123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616118e4-38f4-4e7d-969c-6e97ddf71f68 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:41.840937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:41.843118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.843118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.843373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52936cb1980e4728868f5614b5edefd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.843472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] PATH_INFO: `/v3/projects/ef0d282f4f684b3a82053428e8367905/users/18fe4a9f2f7f43cc9b244b576162d589/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:41.857324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52936cb1980e4728868f5614b5edefd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.866825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0d282f4f684b3a82053428e8367905, user_id=18fe4a9f2f7f43cc9b244b576162d589, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:41.871524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3a9ed-9b1c-4b7b-b2ff-19123d6456ea admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:41.947465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:41.975620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52936cb1980e4728868f5614b5edefd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.976041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52936cb1980e4728868f5614b5edefd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.976660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52936cb1980e4728868f5614b5edefd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:41.982922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 983/1958] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:41 2022] PUT /identity/v3/projects/ef0d282f4f684b3a82053428e8367905/users/18fe4a9f2f7f43cc9b244b576162d589/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:41.993549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 976/1959] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:41 2022] DELETE /identity/v3/users/52936cb1980e4728868f5614b5edefd0 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:41.994932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:41.995287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:41.995627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:42.032789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] MFA Rules not processed for user `18fe4a9f2f7f43cc9b244b576162d589`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:42.034448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:42.129879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 984/1960] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:42.135651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:42.137706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:42.206138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84ed28-3636-4dd0-9cf8-20afc67df210 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:42.210947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:42.211617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:42.211905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:42.212221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:42.213649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:42.215630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:42.397669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 977/1961] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:42 2022] POST /identity/v3/projects => generated 375 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:42.397749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:42.403876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:42.408889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:42.421430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0527b8-be15-47c8-b885-4201b427ff44 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:42.498096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:42.500974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:42.502017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:42.502017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:42.502422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:42.504349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:42.576128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53545446-3187-41e2-b1c5-043d60519dac admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:42.581866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:42.582783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:42.583077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:42.583365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] PATH_INFO: `/v3/projects/f5b345e9c81547e2bdc60edc33bcb157` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:42.584631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b345e9c81547e2bdc60edc33bcb157)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:42.586731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:42.650891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 985/1962] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:42 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:42.651824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:42.653433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:42.667312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb494ed-3e3f-48c4-9a54-2465547d29db admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:42.672926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:42.674577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:42.674870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:42.675154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] PATH_INFO: `/v3/projects/95af44109a354087bfcc33425aba8415` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:42.676383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95af44109a354087bfcc33425aba8415)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:42.678355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:42.734305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b345e9c81547e2bdc60edc33bcb157', 'request_id': 'req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.740866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b345e9c81547e2bdc60edc33bcb157', 'request_id': 'req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.747679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b345e9c81547e2bdc60edc33bcb157', 'request_id': 'req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.870587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95af44109a354087bfcc33425aba8415', 'request_id': 'req-7feec318-8143-4ed8-b49a-2b3f42518955'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.887744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] The token cache is being invalidate because project f5b345e9c81547e2bdc60edc33bcb157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:42.888415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.888926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95af44109a354087bfcc33425aba8415', 'request_id': 'req-7feec318-8143-4ed8-b49a-2b3f42518955'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.889431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:42.891723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 978/1963] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:42 2022] DELETE /identity/v3/projects/f5b345e9c81547e2bdc60edc33bcb157 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:42.893567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:42.909789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:42.997012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae1fd2-51bb-466a-9782-b5c4e0b2a8ae admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.021156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95af44109a354087bfcc33425aba8415', 'request_id': 'req-7feec318-8143-4ed8-b49a-2b3f42518955'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.047276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] The token cache is being invalidate because project 95af44109a354087bfcc33425aba8415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:43.047650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.052689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.064478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.065167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.065459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.065821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.066569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.068188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7a78-0684-47a9-95c0-0eaab089610c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.068711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 986/1964] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:42 2022] DELETE /identity/v3/projects/95af44109a354087bfcc33425aba8415 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:43.069575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.073738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.083463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 979/1965] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:43.085076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.085652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.085842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.112572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feec318-8143-4ed8-b49a-2b3f42518955 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.115995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:43.117219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.121991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.123329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.123768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.124197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] PATH_INFO: `/v3/users/1b815d109a2946388e402b2ea7066fde` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.127975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 980/1966] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:43.128657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b815d109a2946388e402b2ea7066fde)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.136381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.136381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.139418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.153125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf34a29-3581-4a66-880f-9d36ae501bf6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.160940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.162480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.162960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.163917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.166284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.170726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647e4a0-157d-4808-b834-1990541b6e4f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.182200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 981/1967] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:43.185744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.185744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.185744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.210333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:43.210333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.216832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 982/1968] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:43.217905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.229280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18a5e3-315b-4cd3-aced-291ed09a695f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.234366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.235302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.235538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.235816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] PATH_INFO: `/v3/projects/c8ba5342ea3443b1ae1077b5c76478c5/users/31fda3d615e842fe86a8440abf408d21/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.256004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ba5342ea3443b1ae1077b5c76478c5, user_id=31fda3d615e842fe86a8440abf408d21, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.258672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.278924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:43.414288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 983/1969] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:43 2022] PUT /identity/v3/projects/c8ba5342ea3443b1ae1077b5c76478c5/users/31fda3d615e842fe86a8440abf408d21/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:43.415347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.466963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b815d109a2946388e402b2ea7066fde'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.469348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b815d109a2946388e402b2ea7066fde'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.497271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e847e6-5b4c-4688-8807-573813fccb7f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.521890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.521890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.521890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.521890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.523463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:43.524094 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.526243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddab6ab-1fd3-4a3c-a40d-728d53ce6738 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.548205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 984/1970] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:43.548205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.548205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.548573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.587208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] MFA Rules not processed for user `31fda3d615e842fe86a8440abf408d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:43.588919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.655254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b815d109a2946388e402b2ea7066fde'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.655918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b815d109a2946388e402b2ea7066fde'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.667099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 985/1971] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:43.667099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.667099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.668541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b815d109a2946388e402b2ea7066fde'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:43.680538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4afb-6ec3-4753-a00a-ad713781e0a6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.685876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.686655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.686946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.687225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.687930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.689710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 987/1972] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:43 2022] DELETE /identity/v3/users/1b815d109a2946388e402b2ea7066fde => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:43.690319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.693917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.695469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:43.808031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644659f-c03f-4c67-ab66-0c7bd4712020 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.813211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.817446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.817737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.818205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.818920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.820827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:43.889277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 986/1973] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:43.898342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:43.961123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1380-7209-4726-b101-2d8cd091a25e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:43.966588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:43.967266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:43.967579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:43.967861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:43.968590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:43.970706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.117401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 988/1974] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/projects => generated 375 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:44.121855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.176240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 987/1975] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:43 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:44.182501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.208022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ebf9-438d-48d7-9e34-f93a2f7d334f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.251239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0302c0-4f68-4469-b1cf-a3dbbe69ad65 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.261999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.262816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.263179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.263669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.264779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.267886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.319670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.320491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.320854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.322075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.323666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.326795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.354848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 988/1976] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:44.356373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.369858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716481b-2c50-4b13-8555-e43138993be0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.382782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.383547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.384161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.384512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.385321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.387101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.406524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 989/1977] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:44.407912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.418678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 989/1978] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:44 2022] POST /identity/v3/users => generated 294 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:44.420978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c15c3-ab92-4833-975e-f2e3682b6c3b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.425250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.435069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.437055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a487e7-726d-4725-a622-d16924e7c449 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.437342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.438436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.438921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] PATH_INFO: `/v3/projects/19c85ae67e3b4672b8bb2a2299c8a068/users/95bf49afdba147b28ae611c52622b358/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.443052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.444238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.444712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.445200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.446191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.448369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.458534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c85ae67e3b4672b8bb2a2299c8a068, user_id=95bf49afdba147b28ae611c52622b358, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.461125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 990/1979] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:44.463626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.466652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.480078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3197e6-a0f1-41f5-990d-420d4018fbd9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.487917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.489038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.489475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.489942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] PATH_INFO: `/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef/users/ab4515671fcd4168b7295e9bdc24b51c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.510378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f4f825f0e44782b739217c4b6cc8ef, user_id=ab4515671fcd4168b7295e9bdc24b51c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.514540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2994de-3711-4efd-807f-f288fd343817 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.594437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 990/1980] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:44 2022] PUT /identity/v3/projects/19c85ae67e3b4672b8bb2a2299c8a068/users/95bf49afdba147b28ae611c52622b358/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:44.595450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.597391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:44.646485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 991/1981] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:44 2022] PUT /identity/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef/users/ab4515671fcd4168b7295e9bdc24b51c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:44.648556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.648922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.649436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.676174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98faf-c949-4678-a02a-eeea4b899923 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.684003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] MFA Rules not processed for user `95bf49afdba147b28ae611c52622b358`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:44.686206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:44.761199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.762355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.762751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.763141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] PATH_INFO: `/v3/projects/955f78b1798141f5b068e66cbf426145` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.766236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=955f78b1798141f5b068e66cbf426145)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.768870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.796799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 992/1982] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:44.798686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.811559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a26d42-e92f-43b2-a96f-0eaa5a83de08 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.819006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.819824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.820188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.820548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.821483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.823094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.834561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 993/1983] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:44.835527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:44.848014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04959f37-6535-48b4-b65e-0213a4a430ec admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:44.853573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:44.854457 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:44.854818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:44.855175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:44.856234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:44.858256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:44.908560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955f78b1798141f5b068e66cbf426145', 'request_id': 'req-237b6136-ebbe-416d-a4f1-4b281c55ff8c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:44.915970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955f78b1798141f5b068e66cbf426145', 'request_id': 'req-237b6136-ebbe-416d-a4f1-4b281c55ff8c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:44.932995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955f78b1798141f5b068e66cbf426145', 'request_id': 'req-237b6136-ebbe-416d-a4f1-4b281c55ff8c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:44.983088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 994/1984] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:44 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:44.983088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.072247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a9ab-41dd-4ab7-89ce-99b0a3688519 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.083163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.083163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.083163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.083163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] PATH_INFO: `/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef/users/ab4515671fcd4168b7295e9bdc24b51c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.083570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f4f825f0e44782b739217c4b6cc8ef, user_id=ab4515671fcd4168b7295e9bdc24b51c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.084586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.096997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 995/1985] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:44 2022] PUT /identity/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef/users/ab4515671fcd4168b7295e9bdc24b51c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:45.097935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.098499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] The token cache is being invalidate because project 955f78b1798141f5b068e66cbf426145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:45.099828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:45.106503 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-237b6136-ebbe-416d-a4f1-4b281c55ff8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:45.108482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 991/1986] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:44 2022] DELETE /identity/v3/projects/955f78b1798141f5b068e66cbf426145 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:45.113551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6847e-1d21-4c96-a8d5-e1b6826c71f1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.114671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6847e-1d21-4c96-a8d5-e1b6826c71f1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.115067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6847e-1d21-4c96-a8d5-e1b6826c71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.190513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6aa6847e-1d21-4c96-a8d5-e1b6826c71f1 None None] MFA Rules not processed for user `ab4515671fcd4168b7295e9bdc24b51c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:45.191995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6aa6847e-1d21-4c96-a8d5-e1b6826c71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.234853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd2de-4de3-4316-a734-2a8ad07b4815 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.242927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.243956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.244513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.245938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.248100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.250798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.319334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 992/1987] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:45.325696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2682c-87ef-41f4-af91-def759a7766e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.326055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2682c-87ef-41f4-af91-def759a7766e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.326431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2682c-87ef-41f4-af91-def759a7766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.346003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-04f2682c-87ef-41f4-af91-def759a7766e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:45.347333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-04f2682c-87ef-41f4-af91-def759a7766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.355597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 993/1988] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:45.363407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.363701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.363984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.383687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:45.385586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.404058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 994/1989] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:45.408241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.418945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713c051-a252-4c73-bab8-676d71e70b25 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.425235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.426290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.426791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.427148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.428645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:45.429168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.434897 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.449951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 995/1990] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:31:45.453639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.464605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8ec6-8917-4377-83f6-99976480499b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.469443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.470142 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.470430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.470710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.471422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.473338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.497326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:45.510887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 996/1991] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/users => generated 298 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:45.515062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.517159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.532304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0d49c-3be3-4a12-a3b6-f55dc9199aef admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.539833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.548595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.548595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.548595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.548595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.548595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.569115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 997/1992] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:45.570279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.573980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.641525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367970-ec2e-4876-a6a6-29f9137b6cf6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.650692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.651480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.651787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.652072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.654182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:45.692297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 996/1993] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:45.693709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.724046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.725507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.736034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 998/1994] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:45.736872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:45.749395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b072e-c2c0-4dd6-8d66-623aa7fdca5a service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.763526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c861f-0fc4-4650-9602-4a39a474397a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:45.768679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.769666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.770034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.770569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] PATH_INFO: `/v3/projects/b19663e536c14fa7aff8e613480c26b4/users/fd6d5f430aa44275901ca8510be81da1/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.787931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19663e536c14fa7aff8e613480c26b4, user_id=fd6d5f430aa44275901ca8510be81da1, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.792653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:45.817072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:45.818336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:45.818694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:45.819047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:45.819855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:45.821944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.007728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 997/1995] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:45 2022] PUT /identity/v3/projects/b19663e536c14fa7aff8e613480c26b4/users/fd6d5f430aa44275901ca8510be81da1/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:46.011124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 999/1996] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:45 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:46.011983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.012233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.039975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ac1a-1bb7-434c-b0cb-912cc0b3e407 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.122646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.123500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.123860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.124506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.125389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.127172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.136280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64785b19-1b24-44b1-b320-82dc602a7a22 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.141486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.142295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.142590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.143465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.143658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.146343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.156659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1000/1997] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:46.157922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.159427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 998/1998] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:46.163457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.170744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241261db-a03a-4c75-84d4-c8a964c6296c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.175820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9bd8b-63a7-4b64-aaaf-8031525ced5a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.177137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.178340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.178845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.179375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] PATH_INFO: `/v3/projects/b19663e536c14fa7aff8e613480c26b4/users/fd6d5f430aa44275901ca8510be81da1/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.181812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.182795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.183164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.183432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19663e536c14fa7aff8e613480c26b4, user_id=fd6d5f430aa44275901ca8510be81da1, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.183577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.184459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.186228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.186804 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf6c0b-e97c-4d63-8d3f-3f9aa0e66731 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.197824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 999/1999] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:46.202638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.211536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1001/2000] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:46 2022] PUT /identity/v3/projects/b19663e536c14fa7aff8e613480c26b4/users/fd6d5f430aa44275901ca8510be81da1/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:46.213667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a4b6-c9e1-44f5-83c1-eba41555d1c3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.213989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a4b6-c9e1-44f5-83c1-eba41555d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.214267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a4b6-c9e1-44f5-83c1-eba41555d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.216416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a49f83-7908-4c3a-8890-af44c877174b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.246036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9943a4b6-c9e1-44f5-83c1-eba41555d1c3 None None] MFA Rules not processed for user `fd6d5f430aa44275901ca8510be81da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:46.247323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9943a4b6-c9e1-44f5-83c1-eba41555d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:46.290969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.291885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.292171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.292752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] PATH_INFO: `/v3/projects/ae0cd1fee336453681a2c357594628e6/users/12c06a36f146424aace50987cedbf4a5/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.310344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0cd1fee336453681a2c357594628e6, user_id=12c06a36f146424aace50987cedbf4a5, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.313647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.352379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1002/2001] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:46.444363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1000/2002] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:46 2022] PUT /identity/v3/projects/ae0cd1fee336453681a2c357594628e6/users/12c06a36f146424aace50987cedbf4a5/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:46.455494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.456373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.456734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.485529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] MFA Rules not processed for user `12c06a36f146424aace50987cedbf4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:46.486801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:46.559209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1003/2003] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:46.564112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.628609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eb6a-5ee4-420d-8879-32c7bcf42c16 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.633464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.634159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.634470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.634745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.635447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.637432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.722801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.727255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:46.728444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1001/2004] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:46 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:46.729093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.796265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ab3-16b2-4797-ae4d-fb116f8c228b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.798463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f01429-891c-47e5-aeb1-faff014421b0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.801529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.802237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.802659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.802947 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.803647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.805566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.821545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:46.852539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.853797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.854094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.854389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.924110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1002/2005] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:46 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:46.924317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.924317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.925304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.936204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1004/2006] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:46.937755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072af86-a2a8-44dd-83c1-0c269698c437 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.943606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.944222 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.944552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.944836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:46.945625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:46.948263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:46.966422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1003/2007] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:46.970858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:46.982775 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac618b-264c-4eca-b775-92f593d7df08 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:46.987334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:46.988317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:46.988605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:46.988889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] PATH_INFO: `/v3/projects/f2c9ba90c806490284afce160479a875/users/f5eb8a374e674530808e699724039623/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.007986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c9ba90c806490284afce160479a875, user_id=f5eb8a374e674530808e699724039623, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.012073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.118314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1005/2008] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:46 2022] PUT /identity/v3/projects/f2c9ba90c806490284afce160479a875/users/f5eb8a374e674530808e699724039623/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:47.122444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:47.132518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1102c-db81-4436-90bc-cb1b8002a4b4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:47.194229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:47.194918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.195303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.195677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.196489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.198212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaafa6-802c-4a70-a1f2-a18777ab708e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.208545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1004/2009] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:47.213618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:47.232996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3f409-029d-4e43-b0b2-2c4ac5625e64 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:47.240080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:47.241431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.241896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.242315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] PATH_INFO: `/v3/projects/f2c9ba90c806490284afce160479a875/users/f5eb8a374e674530808e699724039623/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.245694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c9ba90c806490284afce160479a875, user_id=f5eb8a374e674530808e699724039623, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.249971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.266915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1006/2010] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:47 2022] PUT /identity/v3/projects/f2c9ba90c806490284afce160479a875/users/f5eb8a374e674530808e699724039623/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:47.280916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.280916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.280916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.314883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] MFA Rules not processed for user `f5eb8a374e674530808e699724039623`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:47.325216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:47.377287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:47.416932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1005/2011] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:47.417866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:47.419146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:47.441287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff6b17-3c42-49c2-a0bb-f578add7881f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:47.445150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:47.445867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.446149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.446451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.449439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:47.486215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fca74-89e0-4c55-82a4-8a5ba874a2f7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:47.491187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:47.492126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.492484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.492833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] PATH_INFO: `/v3/users/8ea9420a857d401db090e89a6bf6a49f` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.494617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea9420a857d401db090e89a6bf6a49f)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.496597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.509676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.511317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.520932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1007/2012] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:47.521788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:47.529687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9858c-af72-40d3-806e-ed43da738896 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:47.534295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:47.534987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.535241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.535675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.545423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:47.546885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da07b58-3faa-494d-ab8f-b62365efd275 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:47.557180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1008/2013] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:47.743583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea9420a857d401db090e89a6bf6a49f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:47.756685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea9420a857d401db090e89a6bf6a49f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:47.895617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea9420a857d401db090e89a6bf6a49f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:47.896361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea9420a857d401db090e89a6bf6a49f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:47.897245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea9420a857d401db090e89a6bf6a49f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:47.913298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1006/2014] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:47 2022] DELETE /identity/v3/users/8ea9420a857d401db090e89a6bf6a49f => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:47.921570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:47.921570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:47.921570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:47.943515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:47.944736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:48.004901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1009/2015] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:48.014345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:48.015819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:48.078657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37864-6cff-4f29-a5ab-2770d5448c62 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:48.082241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:48.082904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:48.083190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:48.083470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:48.094560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:48.096004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:48.105787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1007/2016] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:48.360054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:48.468230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:48.676889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:48.679099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:48.698566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54aad7-a5f3-44a9-a785-4a90fac3d541 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:48.704565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:48.705337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:48.705610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:48.705925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] PATH_INFO: `/v3/projects/b0dc116684ad4fee8b3ba0a83acaf855` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:48.707217 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0dc116684ad4fee8b3ba0a83acaf855)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:48.710364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:48.819715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0dc116684ad4fee8b3ba0a83acaf855', 'request_id': 'req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:48.826448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0dc116684ad4fee8b3ba0a83acaf855', 'request_id': 'req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:48.935796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:48.937273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:48.941285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0dc116684ad4fee8b3ba0a83acaf855', 'request_id': 'req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:48.959163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] The token cache is being invalidate because project b0dc116684ad4fee8b3ba0a83acaf855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:48.959581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:48.960509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:48.962822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1010/2017] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:48 2022] DELETE /identity/v3/projects/b0dc116684ad4fee8b3ba0a83acaf855 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:48.967087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:48.969317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.014546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2765-54b6-4d0d-b6ca-618ef64929eb service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.039944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce31ce-33bb-499c-aaaf-1f7af6cd3252 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.046632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:49.047659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:49.048126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:49.048413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] PATH_INFO: `/v3/users/ce66cf8a13b64d1ab41e881db0558856` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:49.049669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce66cf8a13b64d1ab41e881db0558856)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:49.051900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:49.077040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5eb8a374e674530808e699724039623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c9ba90c806490284afce160479a875', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:49.077733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:49.078225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:49.078776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:49.079693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:49.080089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:49.081809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:49.093732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1008/2018] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:31:49.094588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:49.096312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.152832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbe61e-0d90-4044-b7d7-b5402b4491d9 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.156542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:49.157189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:49.157480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:49.157776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:49.159656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.221008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:49.222524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:49.232480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1009/2019] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:49.234063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:49.235422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.249290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6514-875c-4cf5-adee-160e122c3be3 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.253326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5eb8a374e674530808e699724039623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c9ba90c806490284afce160479a875', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:49.254010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:49.254300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:49.254581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:49.255304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:49.257189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:49.294432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce66cf8a13b64d1ab41e881db0558856'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:49.295123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce66cf8a13b64d1ab41e881db0558856'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:49.403393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1010/2020] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:49 2022] POST /identity/v3/projects => generated 353 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:49.407512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:49.409146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.471825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed308d5e-685b-4636-95f5-6522cc485519 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.476596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5eb8a374e674530808e699724039623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c9ba90c806490284afce160479a875', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:49.477254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:49.477549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:49.488599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:49.488599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:49.488599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:49.488599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:49.493411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1011/2021] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:31:49.527045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce66cf8a13b64d1ab41e881db0558856'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:49.527904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce66cf8a13b64d1ab41e881db0558856'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:49.535956 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce66cf8a13b64d1ab41e881db0558856'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:49.555708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1011/2022] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:48 2022] DELETE /identity/v3/users/ce66cf8a13b64d1ab41e881db0558856 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:49.866078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:49.867973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:49.872840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:49.979713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa331779-c8d6-4833-84f3-ad3c9a70e6df tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.986665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58adb-f998-4f3b-a3a8-aa49f247c5aa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:49.999702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5eb8a374e674530808e699724039623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c9ba90c806490284afce160479a875', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:50.000601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:50.000890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:50.001172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] PATH_INFO: `/v3/projects/8316e8cc75d5485fab015739cadb0e91` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:50.002793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8316e8cc75d5485fab015739cadb0e91)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:50.005047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:50.054657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:50.056224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:50.056525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:50.056805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] PATH_INFO: `/v3/users/45a2bac6401d45888b31aa7b09838ab1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:50.063581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a2bac6401d45888b31aa7b09838ab1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:50.064724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:50.442375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8316e8cc75d5485fab015739cadb0e91', 'request_id': 'req-758fa89d-3847-40c7-8168-7c9152951812'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.442568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a2bac6401d45888b31aa7b09838ab1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.450334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8316e8cc75d5485fab015739cadb0e91', 'request_id': 'req-758fa89d-3847-40c7-8168-7c9152951812'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.455332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a2bac6401d45888b31aa7b09838ab1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.558327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8316e8cc75d5485fab015739cadb0e91', 'request_id': 'req-758fa89d-3847-40c7-8168-7c9152951812'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.569371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] The token cache is being invalidate because project 8316e8cc75d5485fab015739cadb0e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:50.569660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.570321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.572463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1012/2023] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:49 2022] DELETE /identity/v3/projects/8316e8cc75d5485fab015739cadb0e91 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:50.573237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:50.574538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:50.641428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa89d-3847-40c7-8168-7c9152951812 tempest-VolumeQuotaClassesTest-1948708998 tempest-VolumeQuotaClassesTest-1948708998-project-admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:50.646020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:50.646889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:50.647196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:50.647452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] PATH_INFO: `/v3/projects/93f4664de3684d95aba50b83bb554089` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:50.648645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93f4664de3684d95aba50b83bb554089)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:50.650607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:50.683903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a2bac6401d45888b31aa7b09838ab1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.684622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a2bac6401d45888b31aa7b09838ab1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.685254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d030f017-50c3-447a-a926-136c85e430a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a2bac6401d45888b31aa7b09838ab1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.699319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1012/2024] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:49 2022] DELETE /identity/v3/users/45a2bac6401d45888b31aa7b09838ab1 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:50.704108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:50.704413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:50.704703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:50.723035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:50.724269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:50.784662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1013/2025] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:50.794331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:50.804676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:50.821597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f4664de3684d95aba50b83bb554089', 'request_id': 'req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.828049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f4664de3684d95aba50b83bb554089', 'request_id': 'req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.866354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b09e-f8cb-47e7-8aaf-32f95fba42a4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:50.914417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:50.915110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:50.915940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:50.916223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:50.946388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f4664de3684d95aba50b83bb554089', 'request_id': 'req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.958082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] The token cache is being invalidate because project 93f4664de3684d95aba50b83bb554089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:50.958377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.959078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:50.960640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1013/2026] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:50 2022] DELETE /identity/v3/projects/93f4664de3684d95aba50b83bb554089 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:50.964945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:50.966394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:50.977705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:50.979946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:50.991835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1014/2027] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:31:51.035233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8327de5-cd0a-4b43-a449-e1d78a5df4a2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:51.040772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:51.041663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:51.041978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:51.042258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] PATH_INFO: `/v3/users/bdc2d6db110e4b58b0101914d8603899` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:51.043504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc2d6db110e4b58b0101914d8603899)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:51.045441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:51.380948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc2d6db110e4b58b0101914d8603899'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.381610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc2d6db110e4b58b0101914d8603899'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.413362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:51.475554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc2d6db110e4b58b0101914d8603899'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.476212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc2d6db110e4b58b0101914d8603899'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.484576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc2d6db110e4b58b0101914d8603899'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.503288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1014/2028] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:50 2022] DELETE /identity/v3/users/bdc2d6db110e4b58b0101914d8603899 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:51.561057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:51.562862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:51.577122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fae1c-b7e6-47ec-9850-0ae6cb59f574 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:51.638581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:51.639454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:51.639747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:51.640030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] PATH_INFO: `/v3/projects/b0ea3bb43e3649e0b9bcbc62244d91e1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:51.641217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ea3bb43e3649e0b9bcbc62244d91e1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:51.643223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:51.684813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:51.686273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:51.700060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06b22-9a05-4dc8-bbb6-f944630b497c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:51.704625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:51.705492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:51.705808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:51.706247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] PATH_INFO: `/v3/users/95bf49afdba147b28ae611c52622b358` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:51.707461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95bf49afdba147b28ae611c52622b358)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:51.709363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:51.822539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ea3bb43e3649e0b9bcbc62244d91e1', 'request_id': 'req-c2bd1a79-c60f-484e-8cf4-2232c5843784'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.830994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ea3bb43e3649e0b9bcbc62244d91e1', 'request_id': 'req-c2bd1a79-c60f-484e-8cf4-2232c5843784'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:51.839010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ea3bb43e3649e0b9bcbc62244d91e1', 'request_id': 'req-c2bd1a79-c60f-484e-8cf4-2232c5843784'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.004695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] The token cache is being invalidate because project b0ea3bb43e3649e0b9bcbc62244d91e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:52.007270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.007270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.007816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1015/2029] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:51 2022] DELETE /identity/v3/projects/b0ea3bb43e3649e0b9bcbc62244d91e1 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:52.009372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.011047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.067133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95bf49afdba147b28ae611c52622b358'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.067989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95bf49afdba147b28ae611c52622b358'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.088373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd1a79-c60f-484e-8cf4-2232c5843784 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.088373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.088880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.089302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.089595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] PATH_INFO: `/v3/users/12c06a36f146424aace50987cedbf4a5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.090971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c06a36f146424aace50987cedbf4a5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:52.092915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:52.172917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95bf49afdba147b28ae611c52622b358'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.173403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95bf49afdba147b28ae611c52622b358'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.181621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95bf49afdba147b28ae611c52622b358'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.200256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1015/2030] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:51 2022] DELETE /identity/v3/users/95bf49afdba147b28ae611c52622b358 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:52.201117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.204098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.213483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:52.270031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e117004-acd8-430e-b5e5-16b92286e370 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.274813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.275691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.275985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.276268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] PATH_INFO: `/v3/users/c32f2277e4914d95b70fd9810d330361` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.277615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32f2277e4914d95b70fd9810d330361)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:52.279598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:52.329133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c06a36f146424aace50987cedbf4a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.341046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c06a36f146424aace50987cedbf4a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.475565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c06a36f146424aace50987cedbf4a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.476304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c06a36f146424aace50987cedbf4a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.477081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c06a36f146424aace50987cedbf4a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.491135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1016/2031] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:52 2022] DELETE /identity/v3/users/12c06a36f146424aace50987cedbf4a5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:52.492113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.494092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.552548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858834-576a-4abf-bb83-7d34ee5945b2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.556613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.557258 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.557555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.557850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.559599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.578699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32f2277e4914d95b70fd9810d330361'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.582347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32f2277e4914d95b70fd9810d330361'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.622798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:52.624316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:52.634324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1017/2032] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:52.634988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.636498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.683341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32f2277e4914d95b70fd9810d330361'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.683991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32f2277e4914d95b70fd9810d330361'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.691707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-463f257a-3b9d-43e0-80c8-4000c918527d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32f2277e4914d95b70fd9810d330361'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.700290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724044fc-de2f-41d4-a643-ff2256a7a420 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.705674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.706573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.706912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.707195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] PATH_INFO: `/v3/projects/8584ea08db4241668619a8a0bfb4857f` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.708410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8584ea08db4241668619a8a0bfb4857f)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:52.709601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1016/2033] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:52 2022] DELETE /identity/v3/users/c32f2277e4914d95b70fd9810d330361 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:52.710362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:52.711129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687831c-306d-4678-af4f-93116c0acedb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.711472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687831c-306d-4678-af4f-93116c0acedb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.711823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687831c-306d-4678-af4f-93116c0acedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.731698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5687831c-306d-4678-af4f-93116c0acedb None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:52.733241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5687831c-306d-4678-af4f-93116c0acedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.799058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1017/2034] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:52.800535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.800832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.801114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.818693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:52.819966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.828151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1018/2035] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:52.828954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.830214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.839498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8584ea08db4241668619a8a0bfb4857f', 'request_id': 'req-23cf7d71-7371-4e0d-a72b-21beb280f55c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.845605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8584ea08db4241668619a8a0bfb4857f', 'request_id': 'req-23cf7d71-7371-4e0d-a72b-21beb280f55c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.851553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8584ea08db4241668619a8a0bfb4857f', 'request_id': 'req-23cf7d71-7371-4e0d-a72b-21beb280f55c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.886870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948e6e0-4602-4fbc-b212-cea3ba240fac None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.935110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.935769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.936056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.936334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.952250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] The token cache is being invalidate because project 8584ea08db4241668619a8a0bfb4857f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:52.958229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.958229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:52.958229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1018/2036] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:52 2022] DELETE /identity/v3/projects/8584ea08db4241668619a8a0bfb4857f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:52.958229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:52.958229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:52.976330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7d71-7371-4e0d-a72b-21beb280f55c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:52.989957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:52.990858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:52.991154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:52.991574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:52.994415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.015363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.016892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f6b5e-66d8-4693-abdc-6ba87538d662 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.033702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1019/2037] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:53.035947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc3c9-d5b7-46e5-a2f2-0a1f88c794ac None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.036945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc3c9-d5b7-46e5-a2f2-0a1f88c794ac None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.037395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc3c9-d5b7-46e5-a2f2-0a1f88c794ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.074492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c0bdc3c9-d5b7-46e5-a2f2-0a1f88c794ac None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:53.077464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c0bdc3c9-d5b7-46e5-a2f2-0a1f88c794ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.095761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.095761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.098747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1020/2038] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:53.112106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.112668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.113858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.122818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1019/2039] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:53.146151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:53.147895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.170194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1021/2040] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:31:53.176295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.190210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4bb18-dca2-4b34-9ffd-7c0e7997f1a3 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.197421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.198975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.199478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.200029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.201349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:53.201965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.204100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.234966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.239310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1020/2041] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:53.245251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25b75a-5602-42c0-ae1b-b95a0d39e6fc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.256014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.256695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.257150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.257596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.258647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.260767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.322316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.325511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.349831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c61dc8-e7b4-4bff-a578-20ec516981e7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.354651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.355518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.355804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.356348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] PATH_INFO: `/v3/projects/6ccbb85fd6e54c749579e246230cbbb5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.357524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccbb85fd6e54c749579e246230cbbb5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.359472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.427139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1022/2042] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:53 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:53.428244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.460086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:53.501344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e91ea-6272-4500-a5bf-be7657668302 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.507424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.508308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.510124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.510577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.511463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.514047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.547470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccbb85fd6e54c749579e246230cbbb5', 'request_id': 'req-8db7bd88-6898-4932-82f3-eb26177d8cd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:53.556264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccbb85fd6e54c749579e246230cbbb5', 'request_id': 'req-8db7bd88-6898-4932-82f3-eb26177d8cd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:53.562872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccbb85fd6e54c749579e246230cbbb5', 'request_id': 'req-8db7bd88-6898-4932-82f3-eb26177d8cd0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:53.676895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] The token cache is being invalidate because project 6ccbb85fd6e54c749579e246230cbbb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:53.677281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:53.679614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8db7bd88-6898-4932-82f3-eb26177d8cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:53.682328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1023/2044] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:53 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:53.685439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1021/2044] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:53 2022] DELETE /identity/v3/projects/6ccbb85fd6e54c749579e246230cbbb5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:31:53.686068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.687006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.687301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.687588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.688655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.701433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e928-dc0e-437b-a05c-26b3b3f94c8a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.706302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:53.707512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.766228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.767255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.767653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.767891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.768738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.770399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.779055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1022/2045] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:53.780560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.782256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.789823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1024/2046] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:31:53.789823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:53.799761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d9ec-e7b8-4499-a384-92f3d18359fd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.804242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:53.805212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.806450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.806748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.807265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] PATH_INFO: `/v3/projects/19c85ae67e3b4672b8bb2a2299c8a068` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.808520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c85ae67e3b4672b8bb2a2299c8a068)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.810636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.821284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334657-9802-4edb-83fd-902b88d3c870 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:53.826884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:53.827928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:53.828417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:53.828645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] PATH_INFO: `/v3/projects/ae0cd1fee336453681a2c357594628e6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:53.830216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0cd1fee336453681a2c357594628e6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:53.833068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:53.971086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:54.017374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c85ae67e3b4672b8bb2a2299c8a068', 'request_id': 'req-f014bc20-ed1d-4ada-828d-3185647b4a75'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.017481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0cd1fee336453681a2c357594628e6', 'request_id': 'req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.021574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0cd1fee336453681a2c357594628e6', 'request_id': 'req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.023348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c85ae67e3b4672b8bb2a2299c8a068', 'request_id': 'req-f014bc20-ed1d-4ada-828d-3185647b4a75'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.031063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c85ae67e3b4672b8bb2a2299c8a068', 'request_id': 'req-f014bc20-ed1d-4ada-828d-3185647b4a75'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.138592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0cd1fee336453681a2c357594628e6', 'request_id': 'req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.154669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] The token cache is being invalidate because project ae0cd1fee336453681a2c357594628e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:54.154669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.154669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2ee32073-47b2-4ef8-9a2f-1d2be5b1ec85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.154959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1025/2047] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:53 2022] DELETE /identity/v3/projects/ae0cd1fee336453681a2c357594628e6 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:54.156260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.156546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.156822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.174886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:54.176101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.236373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] The token cache is being invalidate because project 19c85ae67e3b4672b8bb2a2299c8a068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:54.236955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.237552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.242317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1023/2048] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:53 2022] DELETE /identity/v3/projects/19c85ae67e3b4672b8bb2a2299c8a068 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:54.242317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.242317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.250573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1026/2049] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:54.251590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.253258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.262275 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014bc20-ed1d-4ada-828d-3185647b4a75 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.268836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.269618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812873-963f-4f26-8d51-e95841a02c84 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.271438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.271962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.272451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.273427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.274957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.275614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.276149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.276400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.276763 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] PATH_INFO: `/v3/users/f5eb8a374e674530808e699724039623` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.278149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5eb8a374e674530808e699724039623)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.280133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.288546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1024/2050] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:54.289641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.293855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.308369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62a6f-6098-4394-9c57-43b923f9a97f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.325347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.329187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.329480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.331421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] PATH_INFO: `/v3/users/fd6d5f430aa44275901ca8510be81da1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.331421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6d5f430aa44275901ca8510be81da1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.333251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.459003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5eb8a374e674530808e699724039623'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.459853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5eb8a374e674530808e699724039623'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.532740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6d5f430aa44275901ca8510be81da1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.545732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6d5f430aa44275901ca8510be81da1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.546681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5eb8a374e674530808e699724039623'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.547346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5eb8a374e674530808e699724039623'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.555071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5eb8a374e674530808e699724039623'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.574114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1027/2051] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:54 2022] DELETE /identity/v3/users/f5eb8a374e674530808e699724039623 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:54.575066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.576436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.644793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6d5f430aa44275901ca8510be81da1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.645444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6d5f430aa44275901ca8510be81da1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.648214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6d5f430aa44275901ca8510be81da1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:54.650022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f638569-f0f0-4d65-acf9-6a02213b14c5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.655059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.655761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.656033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.656317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.657092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:54.657614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.659307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.661532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1025/2052] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:54 2022] DELETE /identity/v3/users/fd6d5f430aa44275901ca8510be81da1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:54.662478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.675312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1028/2053] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:54.676235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.676599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.678561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.693824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031ccf6-5733-4296-bf4c-e709738ff87e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.760204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.760878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.761162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.761453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.762311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.764220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.770449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c7eea-b12a-4be0-9034-5f8b66056784 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:54.776409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:54.777511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.777982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.783033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] PATH_INFO: `/v3/projects/6508b2f4801d4a4ab0494b3201432152/users/f51fdbafd3ec43e28a11f8bf2b94e02b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.796382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6508b2f4801d4a4ab0494b3201432152, user_id=f51fdbafd3ec43e28a11f8bf2b94e02b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:54.799053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaae969-de74-4d51-a128-b365cbb542bb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:54.857819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1029/2054] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:54 2022] POST /identity/v3/projects => generated 401 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:54.861822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:54.878100 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:54.886446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1026/2055] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:54 2022] PUT /identity/v3/projects/6508b2f4801d4a4ab0494b3201432152/users/f51fdbafd3ec43e28a11f8bf2b94e02b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:54.898278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:54.898571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:54.898995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:54.927336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] MFA Rules not processed for user `f51fdbafd3ec43e28a11f8bf2b94e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:54.928570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:54.935996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427909-cb47-4178-8927-9815fb23520e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.005762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.006482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.006776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.007131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.007990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.010067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.038148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1027/2056] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:55.049391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.051365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.113234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1030/2057] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:54 2022] POST /identity/v3/users => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:55.113234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.113234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.116515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6c14c-49d9-42d1-8e55-aec8658ccc05 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.120971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.121744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.122060 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.122335 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.126150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8a19-07d0-48a7-911c-dcf7ab54b3f4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.131086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.131253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.131977 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.132261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.132546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] PATH_INFO: `/v3/projects/f2c9ba90c806490284afce160479a875` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.132687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.134501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c9ba90c806490284afce160479a875)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.136489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.142801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1028/2058] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:55.144316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.155070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c55d-3e99-432f-9bf7-692001bfb4eb service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.162772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.163659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.163951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.164262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.165030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.166681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.180385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1029/2059] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:55.180385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.188044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24372c2-292c-41f9-aa2e-68c15915f54c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.192680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.193523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.193835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.194118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] PATH_INFO: `/v3/users/f51fdbafd3ec43e28a11f8bf2b94e02b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.195314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51fdbafd3ec43e28a11f8bf2b94e02b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.197786 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.331885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c9ba90c806490284afce160479a875', 'request_id': 'req-8b54b857-9c7c-40e7-ab0b-10d3aa243723'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.346783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c9ba90c806490284afce160479a875', 'request_id': 'req-8b54b857-9c7c-40e7-ab0b-10d3aa243723'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.443845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c9ba90c806490284afce160479a875', 'request_id': 'req-8b54b857-9c7c-40e7-ab0b-10d3aa243723'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.460202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] The token cache is being invalidate because project f2c9ba90c806490284afce160479a875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:55.461611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.461723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51fdbafd3ec43e28a11f8bf2b94e02b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.462204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.467291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1031/2060] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:55 2022] DELETE /identity/v3/projects/f2c9ba90c806490284afce160479a875 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:55.467291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.467291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.478392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51fdbafd3ec43e28a11f8bf2b94e02b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.538891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:55.539191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54b857-9c7c-40e7-ab0b-10d3aa243723 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.544118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.544798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1032/2061] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:55.562010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.562545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.575077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfb680-456d-48bb-b975-86f6986e0d9b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.580276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.581235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.581529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.581847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] PATH_INFO: `/v3/projects/b19663e536c14fa7aff8e613480c26b4` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.583089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19663e536c14fa7aff8e613480c26b4)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.585009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.593164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51fdbafd3ec43e28a11f8bf2b94e02b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.595814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51fdbafd3ec43e28a11f8bf2b94e02b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.596132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-56ccf7cd-ae84-4a05-98c3-b983716fd6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51fdbafd3ec43e28a11f8bf2b94e02b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.615821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1030/2062] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:55 2022] DELETE /identity/v3/users/f51fdbafd3ec43e28a11f8bf2b94e02b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:55.617611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.618926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.619588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.640713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:55.642074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.705988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1031/2063] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:55.706744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.708023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.719385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb584a-c814-4018-a0c1-aa87aff2cbaf None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:55.723988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:55.724874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.725159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.725438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] PATH_INFO: `/v3/projects/532b0a29b13b41e0b16b2f487b3376fe/users/72a5460933d44b5f92defabf586ce8a3/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.745018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532b0a29b13b41e0b16b2f487b3376fe, user_id=72a5460933d44b5f92defabf586ce8a3, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:55.747799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f2e04-7cc8-4e38-a0c2-fd90b6c1c1b3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:55.825125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1032/2064] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:55 2022] PUT /identity/v3/projects/532b0a29b13b41e0b16b2f487b3376fe/users/72a5460933d44b5f92defabf586ce8a3/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:55.826584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e3dc-b284-4ad0-9369-f57e44f9d1b3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.826887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e3dc-b284-4ad0-9369-f57e44f9d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.827349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e3dc-b284-4ad0-9369-f57e44f9d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.843617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19663e536c14fa7aff8e613480c26b4', 'request_id': 'req-5c952e70-a308-485e-9961-436c0b8ef827'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.853581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19663e536c14fa7aff8e613480c26b4', 'request_id': 'req-5c952e70-a308-485e-9961-436c0b8ef827'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.853696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dac2e3dc-b284-4ad0-9369-f57e44f9d1b3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:55.854913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dac2e3dc-b284-4ad0-9369-f57e44f9d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.914741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1033/2065] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:55.916551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.916843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.917124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.923697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19663e536c14fa7aff8e613480c26b4', 'request_id': 'req-5c952e70-a308-485e-9961-436c0b8ef827'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.935517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:55.936607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] The token cache is being invalidate because project b19663e536c14fa7aff8e613480c26b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:55.936746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.937029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.937680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c952e70-a308-485e-9961-436c0b8ef827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:55.939398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1033/2066] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:55 2022] DELETE /identity/v3/projects/b19663e536c14fa7aff8e613480c26b4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:55.940705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:55.940918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:55.941199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:55.945019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1034/2067] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:55.945844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:55.947077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:55.968987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] MFA Rules not processed for user `72a5460933d44b5f92defabf586ce8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:55.970164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:56.004237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1ccfd-8d12-4b50-b33d-593996d8e027 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.007600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.008286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.008570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.010877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.016178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:56.027055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.028628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb7b9d-21a1-444e-947b-6d97a4c483da service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.040684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1035/2068] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:56.041528 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.044560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.044560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.064412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:56.073471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:56.074516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1036/2069] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:56.075307 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.077467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1034/2070] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:56.081745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.083261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:56.089062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c73a-75fe-472a-bc6c-0b41abd4116c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.096919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.097642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.098014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.098298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.099049 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:56.103650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.103650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48aaa8-4ba5-4801-8089-f99062f77f73 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.104726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c57108-57b1-4a4f-a61b-3296b18c0878 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.111011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.111855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.112385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.112844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.113679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.116321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1037/2071] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:56.118926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.121906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.122059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.122351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.141791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:56.143095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:56.151843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1038/2072] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:56.156658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.164771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341390b8-c41d-49f8-a37a-d433008fd2b9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.171826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.172502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.172788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.173692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.173827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.175699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.298223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1039/2073] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:56.299361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.302592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1035/2074] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/projects => generated 399 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:31:56.303852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.385745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e82dc-cad2-476e-8f74-4d763b9b595e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.391489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.392244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.392604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.392955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.393369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc088dd4-59e1-4e2a-8e78-a7acc45e1d2a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.393828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.395840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.398283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.400542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.400944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.401293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.402160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:56.402561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.404362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.421684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1040/2075] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:56.421684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.431901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04eace9-651c-400c-8afa-79ceea8f12cf admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.437456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.439593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.439593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.439593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.439593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.441430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.522670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1036/2076] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/users => generated 306 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:56.522670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.532676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec760c84-22f8-4cfa-8a39-12c9154a0778 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.548721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1041/2077] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:56.550522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.550662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.550662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.550662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.550999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.551625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.553557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.560886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c65ccf-adf2-4698-b03e-d634a8394db4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.567012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.571979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.571979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.571979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.571979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.571979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.584157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1042/2078] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:56.585034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.598918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1bbed-a900-4e45-b876-04beb554ca0b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.615789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.615789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.615789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.615789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.616188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.616188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.618622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1043/2079] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:56.619538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.632316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f97a61-34e0-40be-96b7-413a118de1c3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.638951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.639856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.640199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.645945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] PATH_INFO: `/v3/projects/6664cfe04f9740ed8834b1711616e70f/users/c5af0194e23e47f0959dde0ce2296bc0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.660964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6664cfe04f9740ed8834b1711616e70f, user_id=c5af0194e23e47f0959dde0ce2296bc0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.666139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.689424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1037/2080] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/projects => generated 403 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:56.689424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.761962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db177-442d-46cf-9b5f-61b40cbf45ab admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.778013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.778757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.779077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.779369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.780118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.781829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.797021 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1038/2081] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:56.798268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.811387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1044/2082] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:56 2022] PUT /identity/v3/projects/6664cfe04f9740ed8834b1711616e70f/users/c5af0194e23e47f0959dde0ce2296bc0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:56.812209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:56.828115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeabf3-51ec-4e6c-896d-e8b18bbbd6b3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.945639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.946434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.947071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.947071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.948499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.955935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:56.957556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df5ff-0fa8-4a04-9f07-64c208d349d4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:56.963293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:56.964637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:56.965075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:56.965506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] PATH_INFO: `/v3/projects/55911758731f4008b233bef3bd5b8f99/users/4ceb3c64c6cf455e9f657667c01c4e27/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:56.996704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55911758731f4008b233bef3bd5b8f99, user_id=4ceb3c64c6cf455e9f657667c01c4e27, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:56.999541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.083420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1039/2083] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:56 2022] POST /identity/v3/users => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:57.084413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.095059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72b60-7b6c-4989-846c-86b4fa3552e7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.100030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.100686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.101082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.101394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.102151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.104485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.104859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1045/2084] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:56 2022] PUT /identity/v3/projects/55911758731f4008b233bef3bd5b8f99/users/4ceb3c64c6cf455e9f657667c01c4e27/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:57.106266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.107880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.122351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1040/2085] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:57.123263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.135088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23a057-4e3b-47b9-89aa-532a35bfaafa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.168384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:57.179238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfa58-1274-43c4-8ab5-d1e489b2322c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.183933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.184853 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.185137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.185415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] PATH_INFO: `/v3/projects/6508b2f4801d4a4ab0494b3201432152` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.186654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6508b2f4801d4a4ab0494b3201432152)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.188565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.199259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.199924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.200208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.200485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.201321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.202917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707dfdc-756f-4bc5-9260-059a2a8b579e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.213480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1041/2086] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:57.214779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.215071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.215789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.246166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] MFA Rules not processed for user `4ceb3c64c6cf455e9f657667c01c4e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:57.247382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.304127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6508b2f4801d4a4ab0494b3201432152', 'request_id': 'req-32797296-0921-4c89-a7ad-5438d19cd1b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.310501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6508b2f4801d4a4ab0494b3201432152', 'request_id': 'req-32797296-0921-4c89-a7ad-5438d19cd1b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.313052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1042/2087] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:57.313979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.316517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6508b2f4801d4a4ab0494b3201432152', 'request_id': 'req-32797296-0921-4c89-a7ad-5438d19cd1b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.327039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32989e0-c23e-4198-9cad-12547fc5014d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.381901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] The token cache is being invalidate because project 6508b2f4801d4a4ab0494b3201432152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:57.384740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.384740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.384985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1046/2088] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:57 2022] DELETE /identity/v3/projects/6508b2f4801d4a4ab0494b3201432152 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:57.385858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.387155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.387653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.388774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.389342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.389799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] PATH_INFO: `/v3/projects/6664cfe04f9740ed8834b1711616e70f/users/c5af0194e23e47f0959dde0ce2296bc0/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.393848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6664cfe04f9740ed8834b1711616e70f, user_id=c5af0194e23e47f0959dde0ce2296bc0, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.396866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.401632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32797296-0921-4c89-a7ad-5438d19cd1b1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.419196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1043/2089] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:57 2022] PUT /identity/v3/projects/6664cfe04f9740ed8834b1711616e70f/users/c5af0194e23e47f0959dde0ce2296bc0/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:57.419196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.419196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.486458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.487168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.490662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.490662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.490662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.490662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcbb3b-2f5b-4d4f-98be-0cf139012187 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.494342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4fb33-a8c6-487f-bfdb-6987a3dc4246 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.499132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.499697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.499985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.500312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.500993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.504529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1047/2090] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:57.504657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.504738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d339c-7b98-4aa9-a735-cc828a447892 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.504738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d339c-7b98-4aa9-a735-cc828a447892 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.504738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d339c-7b98-4aa9-a735-cc828a447892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.522800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-197d339c-7b98-4aa9-a735-cc828a447892 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:57.524053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-197d339c-7b98-4aa9-a735-cc828a447892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.532351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1048/2091] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:57.533649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.533973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.534255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.544250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:57.564332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] MFA Rules not processed for user `c5af0194e23e47f0959dde0ce2296bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:57.565560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.614070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1044/2092] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:57.614070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.614070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.635418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1049/2093] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:57.636260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.648225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21cdc1-421b-4121-ab8e-5d956fb3bc12 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.678432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70699c9a-3db3-40f7-b19b-5d94078cbd1e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.684475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.685375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.685646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.685960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] PATH_INFO: `/v3/users/18fe4a9f2f7f43cc9b244b576162d589` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.687267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18fe4a9f2f7f43cc9b244b576162d589)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.689187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.708381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.708381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.708381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.708381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] PATH_INFO: `/v3/projects/06c5fb76b95243af9a6fd73f18234ed7/users/20f4fcdc2f6d4761a9e0c14e85e23a9e/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.728714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c5fb76b95243af9a6fd73f18234ed7, user_id=20f4fcdc2f6d4761a9e0c14e85e23a9e, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.731740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b6a7-2dbf-4b9f-81d3-1951d642ff29 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:57.815487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1050/2094] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:57 2022] PUT /identity/v3/projects/06c5fb76b95243af9a6fd73f18234ed7/users/20f4fcdc2f6d4761a9e0c14e85e23a9e/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:57.816854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.817211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.817575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.836312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:57.838165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.891908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18fe4a9f2f7f43cc9b244b576162d589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.892577 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18fe4a9f2f7f43cc9b244b576162d589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:57.907492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1051/2095] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:57.908428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:57.909714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:57.922048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d2a4b-3ba8-4cba-8c99-57b750144bf2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:57.926808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:57.927479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:57.927763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:57.928039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:57.928721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:57.931490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.026890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18fe4a9f2f7f43cc9b244b576162d589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:58.027538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18fe4a9f2f7f43cc9b244b576162d589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:58.039251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-682de048-3467-435b-ade1-5baa31210c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18fe4a9f2f7f43cc9b244b576162d589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:58.054789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1052/2096] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:57 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:58.055670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.066722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.067851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1045/2097] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:57 2022] DELETE /identity/v3/users/18fe4a9f2f7f43cc9b244b576162d589 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:58.072449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.077461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.077461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.108784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] MFA Rules not processed for user `20f4fcdc2f6d4761a9e0c14e85e23a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:58.110131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.140136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c666-d2b5-4e54-a9aa-34f4b136502a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.187746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1046/2098] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:58.188786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.194197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.194895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.195510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.195795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.198539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.261174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54023bc3-2474-4cd3-861d-f6fe500e71ab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.265957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.266415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.267786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.267786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.267786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.268035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.268121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:31:58.268202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.269897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acb37d-679e-4c4f-98d2-b0526c5e574b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.279238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1053/2099] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:58.283573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.283747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1047/2100] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:31:58.283747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.284013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.284278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.291994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6249d-4730-4596-817c-bbe8459bb2d6 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.296657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.297665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.297988 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.298266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.299417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.300765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.305203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:58.308721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.312139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1054/2101] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:58.313005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.317716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.317869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1048/2102] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:58.317869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.326011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0ad3e-9732-48c4-9c0d-90a26e5495e9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.331799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f731-0f10-4930-bd38-fccf0c0db837 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.331931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.331931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.331931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.332248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.332733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.334730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.335101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.335749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.336035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.336313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.337739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.339654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.439960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1049/2103] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:58.439960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.467467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1055/2104] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/projects => generated 407 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:58.468632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.470827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.508834 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc4e58-30c2-4536-8c74-9d90a24c45fb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.529401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2f48-24bc-4f56-a150-43e23b9b452a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.533308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.533981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.534295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.534573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.566252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.567205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.567502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.567786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] PATH_INFO: `/v3/projects/c3d8527d88944f6390e1e40f7ef456cf/users/bc7e81e9947b4a23861676756e121c9f/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.583061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d8527d88944f6390e1e40f7ef456cf, user_id=bc7e81e9947b4a23861676756e121c9f, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.585988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.595972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.597408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.607756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1056/2105] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:58.608535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.617726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b02b7-d808-440b-a6eb-f83ade46a1cf service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.623292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.623957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.624245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.624537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.625443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.627657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.730235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1050/2106] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:58 2022] PUT /identity/v3/projects/c3d8527d88944f6390e1e40f7ef456cf/users/bc7e81e9947b4a23861676756e121c9f/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:58.731318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.759512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1057/2107] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:58.760378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.800091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae2d3c-046f-484f-8eb1-68bd4f6e68ba admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.805757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.806472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.806766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.807056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.807768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.809713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.820891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d4ef-b852-4ab8-85af-7948119505bc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.875917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.876576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.876871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.877149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.879089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:58.945420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1051/2108] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:31:58 2022] POST /identity/v3/users => generated 310 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:31:58.947073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.959386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.960698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3ff9-ee0c-482b-8301-629be950b3d6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.960887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.965848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:58.971303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:58.971303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:58.971303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:58.971303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:58.971303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:58.972215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1058/2109] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:58.974700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.992273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1052/2110] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:58.992273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:58.992826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891762b4-c5a8-4e62-a2c1-8c4e1e8fc9cc service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:58.998505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.000279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.000279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.000279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.001059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.001306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65dee-d3ec-4960-a272-90e9e8d43168 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.002634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.006560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.007425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.007722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.008053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.009531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.011305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.014285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1059/2111] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:59.018281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.025041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1053/2112] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:59.029240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.031951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2ea94-7747-423d-bd8e-6fd43da40ea6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.038242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.039148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.039488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.040397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] PATH_INFO: `/v3/projects/c3d8527d88944f6390e1e40f7ef456cf/users/bc7e81e9947b4a23861676756e121c9f/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.042588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5918f-1f75-4ac0-9931-16477f2f105c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.043978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d8527d88944f6390e1e40f7ef456cf, user_id=bc7e81e9947b4a23861676756e121c9f, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.051573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.051675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.051675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.051675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.051675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.051963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.051963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f4b6-ae0b-4605-af2b-ed8637cc1600 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.069883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1060/2113] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:59 2022] PUT /identity/v3/projects/c3d8527d88944f6390e1e40f7ef456cf/users/bc7e81e9947b4a23861676756e121c9f/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:59.071377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.097414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1054/2114] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:59.097414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.097414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.097414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.143450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] MFA Rules not processed for user `bc7e81e9947b4a23861676756e121c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:59.144846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.193242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece486-837b-4ee0-8a6e-f7567e7efe30 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.224157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.225207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.225554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.225875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] PATH_INFO: `/v3/projects/e432951951204b1499b5d97cc2114e5c/users/f32252adea564651b301b68033880c2a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.251241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1055/2115] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:59.251424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.252756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.256590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e432951951204b1499b5d97cc2114e5c, user_id=f32252adea564651b301b68033880c2a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.259372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.264807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:59.271166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5eb3-9176-4ae2-b591-937460298d88 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.276692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.277675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.278044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.278388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] PATH_INFO: `/v3/projects/ef0d282f4f684b3a82053428e8367905` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.279736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0d282f4f684b3a82053428e8367905)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.281938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.381496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1061/2116] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:59 2022] PUT /identity/v3/projects/e432951951204b1499b5d97cc2114e5c/users/f32252adea564651b301b68033880c2a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:59.382421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.426386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0d282f4f684b3a82053428e8367905', 'request_id': 'req-8f269604-458d-4299-a4be-22bd2b9681e5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.433285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0d282f4f684b3a82053428e8367905', 'request_id': 'req-8f269604-458d-4299-a4be-22bd2b9681e5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.453899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3c6af-af78-47db-9665-734fe26dd097 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.510790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.511685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.511961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.512238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] PATH_INFO: `/v3/projects/69517203e34a4f5eb7ceead9abacc32c/users/425d7d2362f04e12979f2b7b81bbf36f/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.522150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0d282f4f684b3a82053428e8367905', 'request_id': 'req-8f269604-458d-4299-a4be-22bd2b9681e5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.527247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69517203e34a4f5eb7ceead9abacc32c, user_id=425d7d2362f04e12979f2b7b81bbf36f, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.530067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.540624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] The token cache is being invalidate because project ef0d282f4f684b3a82053428e8367905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:31:59.541119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.542170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.544701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1056/2117] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:31:59 2022] DELETE /identity/v3/projects/ef0d282f4f684b3a82053428e8367905 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:59.546205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.548152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.563287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269604-458d-4299-a4be-22bd2b9681e5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.567912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.568568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.568861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.569139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.569910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.572032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.582147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1057/2118] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:31:59.583009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.584660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.640896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3dc0e-4946-4c32-a616-a8d7fab23c9b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.644541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.645188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.645479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.645759 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.647694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.656121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1062/2119] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:59 2022] PUT /identity/v3/projects/69517203e34a4f5eb7ceead9abacc32c/users/425d7d2362f04e12979f2b7b81bbf36f/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:59.657059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.659020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.709279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:31:59.715414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.716869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.727393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1058/2120] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:31:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:31:59.728326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:31:59.735037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a859-5552-49fb-8653-beae0bb1bdf3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.738590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde6a5-9ab5-491b-b6b6-a6a99b545b4d service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:31:59.740121 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.741044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.741331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.741750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] PATH_INFO: `/v3/users/31fda3d615e842fe86a8440abf408d21` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.743560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fda3d615e842fe86a8440abf408d21)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.744855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:31:59.745499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.745736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.747860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.747860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] PATH_INFO: `/v3/projects/e432951951204b1499b5d97cc2114e5c/users/f32252adea564651b301b68033880c2a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.748341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e432951951204b1499b5d97cc2114e5c, user_id=f32252adea564651b301b68033880c2a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:31:59.751094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d863-cd6c-4343-bed5-8c8bde0e1b4c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:31:59.762099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1059/2121] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:31:59 2022] PUT /identity/v3/projects/e432951951204b1499b5d97cc2114e5c/users/f32252adea564651b301b68033880c2a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:31:59.763352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fd0f-434d-46f7-aa05-133186abee67 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.763646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fd0f-434d-46f7-aa05-133186abee67 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.763928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fd0f-434d-46f7-aa05-133186abee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.792794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6a45fd0f-434d-46f7-aa05-133186abee67 None None] MFA Rules not processed for user `425d7d2362f04e12979f2b7b81bbf36f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:59.796736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6a45fd0f-434d-46f7-aa05-133186abee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.873397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1060/2122] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:31:59.874831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619421-dd12-4053-921f-039a409452e9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:31:59.875131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619421-dd12-4053-921f-039a409452e9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:31:59.875419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619421-dd12-4053-921f-039a409452e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:31:59.919848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ad619421-dd12-4053-921f-039a409452e9 None None] MFA Rules not processed for user `f32252adea564651b301b68033880c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:31:59.919848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ad619421-dd12-4053-921f-039a409452e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:31:59.985026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fda3d615e842fe86a8440abf408d21'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:31:59.999723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fda3d615e842fe86a8440abf408d21'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.005345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1061/2123] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:00.010472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619421-dd12-4053-921f-039a409452e9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.011922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ad619421-dd12-4053-921f-039a409452e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.073500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fda3d615e842fe86a8440abf408d21'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.073653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fda3d615e842fe86a8440abf408d21'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.074300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fda3d615e842fe86a8440abf408d21'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.081019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619421-dd12-4053-921f-039a409452e9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.085507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.086306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.086476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.086793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.089438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.091096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1063/2124] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:31:59 2022] DELETE /identity/v3/users/31fda3d615e842fe86a8440abf408d21 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:00.100996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.102470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.111939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1062/2125] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:00.130343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.130343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.135631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.137051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.205608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4a31-726f-4699-b4b9-e5f571bc2f2b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.210330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.211484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.212845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.213330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.215344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.226481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1b4b-eb5f-4b54-b3fc-18a99ec31ee2 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.283801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.284936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.285192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.286019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.286301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.286586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] PATH_INFO: `/v3/users/425d7d2362f04e12979f2b7b81bbf36f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.287795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425d7d2362f04e12979f2b7b81bbf36f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.289746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.295648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1064/2126] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:00.533645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425d7d2362f04e12979f2b7b81bbf36f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.534346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425d7d2362f04e12979f2b7b81bbf36f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.572769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.574401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.590736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537f6d-7445-419b-a0e8-df8223a3c92a service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.595610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.598705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.599014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.599300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] PATH_INFO: `/v3/projects/c8ba5342ea3443b1ae1077b5c76478c5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.601717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ba5342ea3443b1ae1077b5c76478c5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.603736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.669378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425d7d2362f04e12979f2b7b81bbf36f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.670078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425d7d2362f04e12979f2b7b81bbf36f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.677933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1456e128-d3ed-4aad-8035-782038a74545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425d7d2362f04e12979f2b7b81bbf36f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.695694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1063/2127] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:00 2022] DELETE /identity/v3/users/425d7d2362f04e12979f2b7b81bbf36f => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:00.700515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.700815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.701103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.721060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:00.721632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.733154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ba5342ea3443b1ae1077b5c76478c5', 'request_id': 'req-5a418b07-3234-4c6d-ada6-3c10443376f7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.740710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ba5342ea3443b1ae1077b5c76478c5', 'request_id': 'req-5a418b07-3234-4c6d-ada6-3c10443376f7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.747446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ba5342ea3443b1ae1077b5c76478c5', 'request_id': 'req-5a418b07-3234-4c6d-ada6-3c10443376f7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.785835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1064/2128] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:00.787423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.788848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.829568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] The token cache is being invalidate because project c8ba5342ea3443b1ae1077b5c76478c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:00.829941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.831015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:00.832493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1065/2129] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:00 2022] DELETE /identity/v3/projects/c8ba5342ea3443b1ae1077b5c76478c5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:00.836313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.837744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.860720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cd68f-b2c1-4ed7-ad13-193c0d0497ad None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.867097 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.867941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.868891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.869281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.879163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.915480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a418b07-3234-4c6d-ada6-3c10443376f7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:00.924401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:00.925199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:00.925799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:00.926024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:00.928269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.943051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.944624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.958649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1066/2130] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:00.959784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:00.960216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:00.965300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:00.976489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:00.988680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1065/2131] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:00.997104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9b2e0-7f37-4b62-8f9f-d088b7f59169 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:01.000575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:01.001539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:01.001976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:01.002375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] PATH_INFO: `/v3/users/ab4515671fcd4168b7295e9bdc24b51c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:01.003825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4515671fcd4168b7295e9bdc24b51c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:01.006333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:01.012992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:01.348674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4515671fcd4168b7295e9bdc24b51c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:01.360855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4515671fcd4168b7295e9bdc24b51c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:01.507382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4515671fcd4168b7295e9bdc24b51c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:01.508297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4515671fcd4168b7295e9bdc24b51c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:01.509802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4515671fcd4168b7295e9bdc24b51c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:01.532717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1067/2132] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:00 2022] DELETE /identity/v3/users/ab4515671fcd4168b7295e9bdc24b51c => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:01.724044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:01.866795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:01.870704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:01.913243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ccb1b-7120-4234-83e0-1c95972b9222 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:01.995640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:01.998010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.002099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.002407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] PATH_INFO: `/v3/projects/69517203e34a4f5eb7ceead9abacc32c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.002872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.005442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69517203e34a4f5eb7ceead9abacc32c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.006822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.012635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.025816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135672-b4bb-46b2-982f-93299bd6ec56 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.032056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.033761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.034553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.035328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] PATH_INFO: `/v3/users/4ceb3c64c6cf455e9f657667c01c4e27` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.036864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ceb3c64c6cf455e9f657667c01c4e27)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.046761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.207008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69517203e34a4f5eb7ceead9abacc32c', 'request_id': 'req-4e8bd32f-b273-4175-a7de-5672765155d6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.213269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69517203e34a4f5eb7ceead9abacc32c', 'request_id': 'req-4e8bd32f-b273-4175-a7de-5672765155d6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.353565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69517203e34a4f5eb7ceead9abacc32c', 'request_id': 'req-4e8bd32f-b273-4175-a7de-5672765155d6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.370051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] The token cache is being invalidate because project 69517203e34a4f5eb7ceead9abacc32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:02.371160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.372522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ceb3c64c6cf455e9f657667c01c4e27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.372662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.375801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1066/2133] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:01 2022] DELETE /identity/v3/projects/69517203e34a4f5eb7ceead9abacc32c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:02.377485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.380856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.391922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ceb3c64c6cf455e9f657667c01c4e27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.460858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd32f-b273-4175-a7de-5672765155d6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.466517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.467913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.468248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.468491 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] PATH_INFO: `/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.469756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f4f825f0e44782b739217c4b6cc8ef)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.471736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.490392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ceb3c64c6cf455e9f657667c01c4e27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.491686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ceb3c64c6cf455e9f657667c01c4e27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.493344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ce71c7ff-31da-48a3-be15-ec2fc81522cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ceb3c64c6cf455e9f657667c01c4e27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.515085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1068/2134] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:02 2022] DELETE /identity/v3/users/4ceb3c64c6cf455e9f657667c01c4e27 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:02.515085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b6dce-671d-4803-9b94-875ba12e2fe2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.515833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b6dce-671d-4803-9b94-875ba12e2fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.516562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b6dce-671d-4803-9b94-875ba12e2fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.539597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d16b6dce-671d-4803-9b94-875ba12e2fe2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:02.540879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d16b6dce-671d-4803-9b94-875ba12e2fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.597072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f4f825f0e44782b739217c4b6cc8ef', 'request_id': 'req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.604505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f4f825f0e44782b739217c4b6cc8ef', 'request_id': 'req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.605266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1069/2135] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:02.606722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72dbc1-e4de-46c1-a4da-32869824b59e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.607024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72dbc1-e4de-46c1-a4da-32869824b59e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.607460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72dbc1-e4de-46c1-a4da-32869824b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.626708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bd72dbc1-e4de-46c1-a4da-32869824b59e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:02.628023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bd72dbc1-e4de-46c1-a4da-32869824b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.700790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1070/2136] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:02.702376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476464d-18dd-4804-8db0-7da117e3218f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.702799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476464d-18dd-4804-8db0-7da117e3218f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.703090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476464d-18dd-4804-8db0-7da117e3218f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.706779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f4f825f0e44782b739217c4b6cc8ef', 'request_id': 'req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.724455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] The token cache is being invalidate because project e4f4f825f0e44782b739217c4b6cc8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:02.725524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.726623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:02.730006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1067/2137] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:02 2022] DELETE /identity/v3/projects/e4f4f825f0e44782b739217c4b6cc8ef => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:02.733808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.733808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.739109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9476464d-18dd-4804-8db0-7da117e3218f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:02.740584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9476464d-18dd-4804-8db0-7da117e3218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.752973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1071/2138] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:02.754933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.755408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.755959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.786269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:02.787514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.815100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1072/2139] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:02.815998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.826255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b714e8-db8d-43e7-aa72-6f74e15bfe18 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.832758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.833447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.833741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.834055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.835459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:02.835459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.837348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6fb2-0e87-4af2-9a54-ab76d7e484d1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.854965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6dcf3-a788-4ddf-b171-dc31b3bce75d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.855163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1073/2140] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:02.855163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.855163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.855163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.859165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.860002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.860452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.864987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.864987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.888155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.889656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.902244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:02.916524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:02.927322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1068/2141] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:02.930302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1074/2142] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:02.931300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.935108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:02.950099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2eb89-0740-469b-bf0b-b330cc0ac2c7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.968992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f748c-e268-425f-b521-3b5393ccc9b1 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:02.976550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.977344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.978200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.978483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.988868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:02.988868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:02.988868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:02.995255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:02.996427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:02.996918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:02.997403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:02.998469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.013963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.014432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1075/2143] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:32:03.018219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.039956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5072592-2597-46d0-a411-97c3b545c734 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.069855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.069855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.069855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.069855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.070387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.073743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.202905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1069/2144] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:02 2022] POST /identity/v3/projects => generated 361 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:03.212450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.256122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1076/2145] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:03.265965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.339110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739447-e28e-49f7-bf07-0148ec1857cf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.344579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:03.347762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.353169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.358542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5b80-4fd8-4319-90db-01dd53f054a9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.376471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.377262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.377565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.378369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.379350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.381542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.510742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1070/2146] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:03 2022] POST /identity/v3/users => generated 281 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:03.514016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.532861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6d78-7d5b-47c0-a89f-86a632d640da admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.542401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.543139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.543438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.543724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.544605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.546737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.559074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1071/2147] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:03.563224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.574275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edae464-f0b3-4ce9-903d-6aad1207ccff admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.579461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.580197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.580541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.580877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.590295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.590437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1077/2148] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:03 2022] POST /identity/v3/users => generated 287 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:03.590537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.593352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.598392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1072/2149] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:03.600053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.604869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a9f1-e9d4-4a73-8561-79295409bdb5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.610290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.611062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.611410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.611754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.611849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf21059-1134-400c-bcfd-cf7149cd1816 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.612623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.614705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.629201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1078/2150] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:03.630264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.631757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.632175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:03.633234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.635257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.635626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] PATH_INFO: `/v3/projects/ba0419dda3eb44c1bb2f3ecdddd7e110/users/3364a7bdfbde4951bc52830a0bb1a163/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.651833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3ee7-be84-452d-a776-2515f94476c2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.657434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.658635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.658881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.659297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] PATH_INFO: `/v3/projects/55911758731f4008b233bef3bd5b8f99` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.660722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=55911758731f4008b233bef3bd5b8f99)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.662830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.684225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0419dda3eb44c1bb2f3ecdddd7e110, user_id=3364a7bdfbde4951bc52830a0bb1a163, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.687212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.781810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1073/2151] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:03 2022] PUT /identity/v3/projects/ba0419dda3eb44c1bb2f3ecdddd7e110/users/3364a7bdfbde4951bc52830a0bb1a163/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:03.782691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:03.854724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55911758731f4008b233bef3bd5b8f99', 'request_id': 'req-b11555dc-6b3e-4b7b-9608-123c0778c40a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:03.858582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80875ecd-a23b-4a72-8130-19d7d4836914 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:03.864914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55911758731f4008b233bef3bd5b8f99', 'request_id': 'req-b11555dc-6b3e-4b7b-9608-123c0778c40a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:03.886243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55911758731f4008b233bef3bd5b8f99', 'request_id': 'req-b11555dc-6b3e-4b7b-9608-123c0778c40a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:03.929894 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:03.930615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.930913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.931194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.932127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:03.933851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf283a1-bf75-49ea-baf0-2f2a8f419ca7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:03.947916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1074/2152] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:03.947916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:03.947916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:03.947916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:03.989632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] MFA Rules not processed for user `3364a7bdfbde4951bc52830a0bb1a163`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:03.991188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.004368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] The token cache is being invalidate because project 55911758731f4008b233bef3bd5b8f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:04.009725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.010798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.016216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1079/2153] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:03 2022] DELETE /identity/v3/projects/55911758731f4008b233bef3bd5b8f99 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:04.020450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.022059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.043842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11555dc-6b3e-4b7b-9608-123c0778c40a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:04.048811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:04.049933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.052841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.052841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] PATH_INFO: `/v3/projects/6b36d14d5bd24ed39bcb0a32dfa016f7/users/f86340c271b64c14b30834c51a477ca6/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.067397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b36d14d5bd24ed39bcb0a32dfa016f7, user_id=f86340c271b64c14b30834c51a477ca6, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:04.070128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:04.078032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1075/2154] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:04.078888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.080521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.099191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a17eee-80a7-42c4-971d-d2aba0acd8c9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:04.099628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:04.103926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:04.104865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.105201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.105484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] PATH_INFO: `/v3/users/bc7e81e9947b4a23861676756e121c9f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.112379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7e81e9947b4a23861676756e121c9f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:04.112379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:04.199188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1080/2155] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:04 2022] PUT /identity/v3/projects/6b36d14d5bd24ed39bcb0a32dfa016f7/users/f86340c271b64c14b30834c51a477ca6/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:04.199810 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.201647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.283972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe89bd-3999-46d6-a2e7-ff45207276cc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:04.287775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:04.300068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.300068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.300068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.308158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7e81e9947b4a23861676756e121c9f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.308684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7e81e9947b4a23861676756e121c9f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.365138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:04.366909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c899f5-78a9-41c9-abab-62b1341edb6a service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:04.378114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1081/2156] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:04.379728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.380208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.380702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.395016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7e81e9947b4a23861676756e121c9f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.395016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7e81e9947b4a23861676756e121c9f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.404776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:04.407500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.411673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7e81e9947b4a23861676756e121c9f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.437756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1076/2157] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:04 2022] DELETE /identity/v3/users/bc7e81e9947b4a23861676756e121c9f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:04.437756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.437756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.533805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79bfd-a511-4636-810e-2e70e1bbf794 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:04.541863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:04.543413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.544174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1082/2158] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:04.544316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.544677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] PATH_INFO: `/v3/users/3364a7bdfbde4951bc52830a0bb1a163` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.547098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3364a7bdfbde4951bc52830a0bb1a163)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:04.551107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:04.559414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.561689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.647259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7603d4-e8d5-4a97-a5b0-6bc19b1b5635 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:04.658156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:04.707596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:04.708803 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.709205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.709574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.720916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:04.730587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ededc9-6e8e-43b7-8aa6-6886c3cc59bf service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:04.747143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1083/2159] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:04.781160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3364a7bdfbde4951bc52830a0bb1a163'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.783765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3364a7bdfbde4951bc52830a0bb1a163'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.912641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3364a7bdfbde4951bc52830a0bb1a163'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.913464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3364a7bdfbde4951bc52830a0bb1a163'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.922223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3364a7bdfbde4951bc52830a0bb1a163'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:04.955486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1077/2160] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:04 2022] DELETE /identity/v3/users/3364a7bdfbde4951bc52830a0bb1a163 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:04.956815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:04.957143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:04.957381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:04.971685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:04.978169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:04.979617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:04.981737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.052443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42d63b-a257-422a-910e-315aa01cbcf4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.069265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.077976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.077976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.077976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] PATH_INFO: `/v3/users/72a5460933d44b5f92defabf586ce8a3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.077976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a5460933d44b5f92defabf586ce8a3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.078269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1084/2161] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:05.079469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.092663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:05.094439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.197840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a99c1-a1f9-4848-83c6-a5858ba2aaa3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.202630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.206413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.206413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.206611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.239671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.240968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.257995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1085/2162] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:05.259254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:05.260831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.277901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89745d16-f03b-4f97-845b-4b3f874c4815 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.284140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.285125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.285518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.285956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] PATH_INFO: `/v3/projects/c3d8527d88944f6390e1e40f7ef456cf` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.287761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d8527d88944f6390e1e40f7ef456cf)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.290563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.307454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:05.432787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a5460933d44b5f92defabf586ce8a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.438369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a5460933d44b5f92defabf586ce8a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.457683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d8527d88944f6390e1e40f7ef456cf', 'request_id': 'req-04482d7b-ce57-478d-9b60-655a0326978c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.468998 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d8527d88944f6390e1e40f7ef456cf', 'request_id': 'req-04482d7b-ce57-478d-9b60-655a0326978c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.481338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d8527d88944f6390e1e40f7ef456cf', 'request_id': 'req-04482d7b-ce57-478d-9b60-655a0326978c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.527234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a5460933d44b5f92defabf586ce8a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.527919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a5460933d44b5f92defabf586ce8a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.537428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-da1a87ef-74f9-4b47-b64c-97dd2ec7041b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a5460933d44b5f92defabf586ce8a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.562157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1078/2163] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:04 2022] DELETE /identity/v3/users/72a5460933d44b5f92defabf586ce8a3 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:05.566650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ef69-17d8-4d57-8812-530946e9c750 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.567017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ef69-17d8-4d57-8812-530946e9c750 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.567289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ef69-17d8-4d57-8812-530946e9c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.592465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ff89ef69-17d8-4d57-8812-530946e9c750 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:05.593799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ff89ef69-17d8-4d57-8812-530946e9c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.682393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] The token cache is being invalidate because project c3d8527d88944f6390e1e40f7ef456cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:05.682815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.687206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1079/2164] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:05.687323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:05.687323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1086/2165] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:05 2022] DELETE /identity/v3/projects/c3d8527d88944f6390e1e40f7ef456cf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:05.693538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7750-0edf-491f-9df2-81ef4c462f13 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.693980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7750-0edf-491f-9df2-81ef4c462f13 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.694367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7750-0edf-491f-9df2-81ef4c462f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.698467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:05.699594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.745952 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-21af7750-0edf-491f-9df2-81ef4c462f13 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:05.747315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21af7750-0edf-491f-9df2-81ef4c462f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.767377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1080/2166] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:05.769036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.769435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.770651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.794634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:05.796026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.813470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1081/2167] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:05.813470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:05.834411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b2156-4be0-4965-9674-cc195c3f12d7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.840704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04482d7b-ce57-478d-9b60-655a0326978c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.856766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.861423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.861423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.861423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.864699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:05.867906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.871313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.908489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1082/2168] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:05.909592 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:05.926250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88edea-dc0b-4996-b8ff-94af910b2ee7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:05.941617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.942762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.943232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.944047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:05.944441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.944706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:05.944991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:05.945304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:05.945504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.947266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:05.949996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.962859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:05.964832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:05.978440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1087/2169] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:06.149180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1083/2170] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:05 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:06.155772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.178664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b0b8f-8036-4ed2-ba00-bc4211742cf0 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.210856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.214447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.277394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.278254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.278544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.278825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.280699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.283601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.321290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:06.326929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83926d3-81c8-4f34-9941-a32f4f307ce8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.331769 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.332702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.333046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.333329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] PATH_INFO: `/v3/projects/ba0419dda3eb44c1bb2f3ecdddd7e110` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.334591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0419dda3eb44c1bb2f3ecdddd7e110)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.336508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.383585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1088/2171] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:06 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:06.383585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.383585 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.394945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179241e4-c4fe-44cf-b0d1-2039850e1631 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.399527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.400386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.400701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.400980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] PATH_INFO: `/v3/users/e2ac3e31fee448d7ae7a6d885f545636` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.403711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ac3e31fee448d7ae7a6d885f545636)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.404181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.482061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0419dda3eb44c1bb2f3ecdddd7e110', 'request_id': 'req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.489571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0419dda3eb44c1bb2f3ecdddd7e110', 'request_id': 'req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.620905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0419dda3eb44c1bb2f3ecdddd7e110', 'request_id': 'req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.632126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] The token cache is being invalidate because project ba0419dda3eb44c1bb2f3ecdddd7e110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:06.632420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.633044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.634677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1084/2172] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:06 2022] DELETE /identity/v3/projects/ba0419dda3eb44c1bb2f3ecdddd7e110 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:06.635586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.637462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.702519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608fa9f-1c0b-4720-977e-75e4f0a7e6bd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.707220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.707892 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.708180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.708461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.709192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.710879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87692c76-7ce4-4d27-810b-182976b5c39c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.721605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1085/2173] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:06.722891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.723189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.723471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.749172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:06.750793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.760965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1086/2174] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:06.761991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.770964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ac3e31fee448d7ae7a6d885f545636'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.772279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2795939-a610-4acb-ab67-7decb99f21aa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.780896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.781576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.781903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.782189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.782925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.784581 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.785301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ac3e31fee448d7ae7a6d885f545636'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.804461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1087/2175] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:06.804461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:06.806251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.819896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c721-54d9-4255-9466-7c19e85e40c2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:06.827353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:06.828701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.828988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.829268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] PATH_INFO: `/v3/projects/532b0a29b13b41e0b16b2f487b3376fe` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.830783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532b0a29b13b41e0b16b2f487b3376fe)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:06.832704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:06.877217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ac3e31fee448d7ae7a6d885f545636'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.877217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ac3e31fee448d7ae7a6d885f545636'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.877217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b860365a-1839-4b24-b6cc-f40f386c61fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ac3e31fee448d7ae7a6d885f545636'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:06.888774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1089/2176] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:06 2022] DELETE /identity/v3/users/e2ac3e31fee448d7ae7a6d885f545636 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:06.890353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:06.890648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:06.890927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:06.911410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:06.912788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:06.992364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532b0a29b13b41e0b16b2f487b3376fe', 'request_id': 'req-c82f9a92-9e79-4c84-9184-14cc20934e76'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.001912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532b0a29b13b41e0b16b2f487b3376fe', 'request_id': 'req-c82f9a92-9e79-4c84-9184-14cc20934e76'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.002910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1090/2177] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:07.007281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.010007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.022815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecacd4-0191-44b6-a039-13c7008924c8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.094456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.095570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.095928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.096860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] PATH_INFO: `/v3/projects/475c10fd2c5e4245965e4d915b0cc4f2/users/78c0cde810534d29b999ba8ef5dc5adc/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.118449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532b0a29b13b41e0b16b2f487b3376fe', 'request_id': 'req-c82f9a92-9e79-4c84-9184-14cc20934e76'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.125460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475c10fd2c5e4245965e4d915b0cc4f2, user_id=78c0cde810534d29b999ba8ef5dc5adc, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.125669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:07.131505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] The token cache is being invalidate because project 532b0a29b13b41e0b16b2f487b3376fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:07.131817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.132598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.139131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1088/2178] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:06 2022] DELETE /identity/v3/projects/532b0a29b13b41e0b16b2f487b3376fe => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:07.140423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.141683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.167432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f9a92-9e79-4c84-9184-14cc20934e76 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.171341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.172025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.172315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.177025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.177025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:07.177025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.177025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:07.187351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1089/2179] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:07.190228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.192011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.205555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d172c7-cfac-4214-8753-20af6028ec51 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.210532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.211443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.212285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.212570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] PATH_INFO: `/v3/users/c5af0194e23e47f0959dde0ce2296bc0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.213794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5af0194e23e47f0959dde0ce2296bc0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.215720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:07.260656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1091/2180] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:07 2022] PUT /identity/v3/projects/475c10fd2c5e4245965e4d915b0cc4f2/users/78c0cde810534d29b999ba8ef5dc5adc/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:07.260656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.343777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a972-c08a-43d0-a774-7b1cb546d387 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.351040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.352339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.352695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.353455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.354406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.356910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:07.468777 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1092/2181] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:07 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:07.470877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.473103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.479151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5af0194e23e47f0959dde0ce2296bc0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.482549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5af0194e23e47f0959dde0ce2296bc0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.547447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf890958-2f81-4c5e-aa49-ec2693136f4b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.552644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.553588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.554338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.554338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] PATH_INFO: `/v3/projects/8725a19566454345b77b639cfe170008` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.556038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5af0194e23e47f0959dde0ce2296bc0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.556165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8725a19566454345b77b639cfe170008)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.556709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5af0194e23e47f0959dde0ce2296bc0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.558223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:07.568558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6a25a396-6648-40d8-8f71-7dcf66d4e52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5af0194e23e47f0959dde0ce2296bc0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.590766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1090/2182] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:07 2022] DELETE /identity/v3/users/c5af0194e23e47f0959dde0ce2296bc0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:07.592149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.592546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.592866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.625318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] MFA Rules not processed for user `78c0cde810534d29b999ba8ef5dc5adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:07.625318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.697891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1091/2183] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:07.698620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.700717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.728332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8725a19566454345b77b639cfe170008', 'request_id': 'req-c80b4d03-2682-4093-9a46-dc04c8c7b600'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.736433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8725a19566454345b77b639cfe170008', 'request_id': 'req-c80b4d03-2682-4093-9a46-dc04c8c7b600'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.743818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8725a19566454345b77b639cfe170008', 'request_id': 'req-c80b4d03-2682-4093-9a46-dc04c8c7b600'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.798703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b227a51-5ea4-49fe-96f9-5c9fdf78c850 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:07.850570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] The token cache is being invalidate because project 8725a19566454345b77b639cfe170008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:07.853399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.854688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:07.860565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1093/2184] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:07 2022] DELETE /identity/v3/projects/8725a19566454345b77b639cfe170008 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:07.869124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:07.871747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:07.910086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:07.910801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:07.911098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:07.911379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:07.912250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:07.914229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594330aa-0be6-49c1-98f0-01dbdf391d36 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.000822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b4d03-2682-4093-9a46-dc04c8c7b600 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.006404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.007795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.008132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.008442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.009115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.011119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.020761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1092/2185] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:07 2022] POST /identity/v3/users => generated 288 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:08.022131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.022425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.022705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.055284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:08.070320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:08.074908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.098858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:08.109862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1093/2186] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:08.109862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.112073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.128601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4559cd-c7b8-482c-b8a2-01d285afa4ee None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.143549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.146720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.147231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.147579 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.148960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.151849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa0b-cacc-47d0-b5a5-9bcf2b174cda admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.165652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1094/2187] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:08.166510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.167080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.167319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.185860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1094/2188] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:07 2022] POST /identity/v3/projects => generated 385 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:08.187353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.192489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:08.194810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.291000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34de2c-8fec-43a5-80ff-69a703de1145 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.297052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.297831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.298588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.298929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.299754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.302015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.317396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1095/2189] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:08.317396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.331355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4a32f-ef52-493d-8d98-c043595871c9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.337512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1095/2190] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:08.337512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.337512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.340393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.341183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.341573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.342116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.342939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.345128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.348908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456da4f-5b69-4a2b-96d2-bb0a3b9de0ae None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.354335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.355408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.356046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.356400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] PATH_INFO: `/v3/projects/6664cfe04f9740ed8834b1711616e70f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.357696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6664cfe04f9740ed8834b1711616e70f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.360018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.479229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1096/2191] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:08.479229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.488413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851a71-ac54-4989-ab1d-453349668cf2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.493313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.494212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.494504 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.494791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] PATH_INFO: `/v3/projects/f2329de0783643748124e381dfc01084/users/71c27adb4cf3437eaa795efa12deebe6/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.515405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2329de0783643748124e381dfc01084, user_id=71c27adb4cf3437eaa795efa12deebe6, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.519356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.526273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6664cfe04f9740ed8834b1711616e70f', 'request_id': 'req-bfbab2cd-40df-4528-881a-ebb378fa79a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:08.539034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6664cfe04f9740ed8834b1711616e70f', 'request_id': 'req-bfbab2cd-40df-4528-881a-ebb378fa79a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:08.606293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6664cfe04f9740ed8834b1711616e70f', 'request_id': 'req-bfbab2cd-40df-4528-881a-ebb378fa79a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:08.610790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1097/2192] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:08 2022] PUT /identity/v3/projects/f2329de0783643748124e381dfc01084/users/71c27adb4cf3437eaa795efa12deebe6/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:08.611636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.622529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] The token cache is being invalidate because project 6664cfe04f9740ed8834b1711616e70f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:08.622941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:08.623634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:08.626324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1096/2193] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:08 2022] DELETE /identity/v3/projects/6664cfe04f9740ed8834b1711616e70f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:08.626562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.628096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.701602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9700340-a8c5-4e35-a870-14eb155cc306 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.701898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbab2cd-40df-4528-881a-ebb378fa79a0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.707060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.707227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.707749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.708032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.708150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.708394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.708629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.708727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.709702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:08.709927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.710153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.712242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042dff2-d1a3-48b1-a553-b6639cf59410 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.712577 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e4f-0de7-4d38-bbb9-d624e64d23e2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.729380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1097/2194] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:08.729380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17cbe4-5521-4951-9d05-767429bc1353 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.729380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17cbe4-5521-4951-9d05-767429bc1353 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.729380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17cbe4-5521-4951-9d05-767429bc1353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.731355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1098/2195] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:08.732705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.733002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.733497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.755453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:08.756702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.767359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1099/2196] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:08.773603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.773840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ab17cbe4-5521-4951-9d05-767429bc1353 None None] MFA Rules not processed for user `71c27adb4cf3437eaa795efa12deebe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:08.773840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab17cbe4-5521-4951-9d05-767429bc1353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.782739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded9d11-6ff2-4e32-8da2-73b03576bee5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.787973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.788664 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.788952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.789235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.790127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.792008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.803460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1100/2197] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:08.804436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.812107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.842303 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190ec4-c820-4e13-af4a-5db01bd94f84 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.849052 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.850652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.850943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.851221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.851926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.855320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:08.869798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1098/2198] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:08.871140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.871519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.871851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.895779 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:08.897053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:08.910883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1099/2199] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:08.910883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:08.933961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2524d-cfbf-42e7-ad20-720389c028e1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:08.933961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:08.933961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:08.933961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:08.934323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] PATH_INFO: `/v3/projects/aab79e44d3194af6aea35631974dc515/users/08c2c1384c364535bcd3828c3b242af5/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:08.945694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab79e44d3194af6aea35631974dc515, user_id=08c2c1384c364535bcd3828c3b242af5, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:08.949728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.032623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1101/2200] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:08 2022] POST /identity/v3/projects => generated 405 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:09.032623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.035672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:09.085884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1100/2201] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:08 2022] PUT /identity/v3/projects/aab79e44d3194af6aea35631974dc515/users/08c2c1384c364535bcd3828c3b242af5/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:09.101020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.160020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab175bb0-4fd4-41dc-bb36-46569ec1f413 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.190835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa16acc-28bc-4ac0-84f7-a924cd9e4b52 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.193496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:09.199185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.199823 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.200115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.200737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.201640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:09.203651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.204681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.223433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.225381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1101/2202] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:09.226331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.238290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680c0a1-eb1a-41c8-8095-6a48fa36a13d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.240532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.241093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.242124 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.245122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.245898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.246211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.246499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.247218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.249253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c0eaf-846d-4027-96ee-9c54929f3904 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.362350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.363940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.389816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1102/2203] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/users => generated 303 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:09.390832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1102/2204] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:09.390966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.391176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.392007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.393207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.408606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c77375-0d09-4365-8757-9952efbc5c95 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.421939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.422945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.423237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.423580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.426835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.429109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.466644 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] MFA Rules not processed for user `08c2c1384c364535bcd3828c3b242af5`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:09.468076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:09.518501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1103/2205] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:09.519606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.568174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1103/2206] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:09.571653 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.595766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33d7c7-7625-488a-ae56-664e72bf16c7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.667532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b20a5-8d35-4f1e-8cba-54f20c170cc8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.680920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.681800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.693252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.695602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.696447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.698146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.720010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1104/2207] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:09.720010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.728932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.730340 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7747d-1f90-4f7f-aaac-787aea8cefe5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.735481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.737006 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.737301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.739287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.741558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.742550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.743511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.743821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.744387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.748878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.748878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.834633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1104/2208] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/users => generated 291 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:09.835552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.845813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f45ea9-4062-421e-a597-487c31018bf8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.850692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.851354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.851637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.851913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.853211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.854909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.866202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1105/2209] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:09.870874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.876626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1105/2210] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:09.877723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.879093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c307a-e07b-42c3-a3af-a7c890a783e8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.951498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.952216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.952885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.953167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.954216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.956005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:09.969986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1106/2211] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:09.971104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:09.974073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207173a-66c9-4817-8fc7-e7379a441577 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.986438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b6bb-ccfd-4ece-a6a3-a09683652bdc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:09.987842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.989102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.989417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.989698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] PATH_INFO: `/v3/projects/e0b7a80ee40b42ce97debf7dcfec27f9/users/059be17987d64fd7b98beacea54aa2c5/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.991002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:09.991731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:09.992020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:09.992432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:09.993943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:09.995910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.022613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b7a80ee40b42ce97debf7dcfec27f9, user_id=059be17987d64fd7b98beacea54aa2c5, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.026353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.076807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1107/2212] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:09 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:10.076807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.092200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8afb18-b626-43fd-b3c7-89cad99637e2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.099091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.099972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.100334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.100690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.101571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.104077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ef2a6-3e35-4f05-b6ef-6214d5fdab3d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.108558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1106/2213] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:09 2022] PUT /identity/v3/projects/e0b7a80ee40b42ce97debf7dcfec27f9/users/059be17987d64fd7b98beacea54aa2c5/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:10.109552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.126277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1108/2214] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:10.126277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.126277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.126277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.174874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] MFA Rules not processed for user `059be17987d64fd7b98beacea54aa2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:10.176220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:10.218643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c6f0-372a-4a65-87d1-bd4276ac2214 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.232815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.234748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.235071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.237095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.238266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.240532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.244352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:10.264021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1107/2215] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:10.264021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.284017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1109/2216] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:10.298929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d001e-5b26-4d54-b367-c91b5706da7e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.299686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.301467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83e2a8-de8b-4646-a215-b99570ded587 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.302617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.303820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.304251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.304674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] PATH_INFO: `/v3/projects/cb6be285203847918fec55b7564f6d28/users/41353328ca434358acf5c10c7088ce74/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.307632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.308682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.309226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.309611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] PATH_INFO: `/v3/projects/6602649962eb4dfdb6e43b225af1a7e1/users/c4cc5c910a444d41b7c8c94d6a2eea7b/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.329888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6be285203847918fec55b7564f6d28, user_id=41353328ca434358acf5c10c7088ce74, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.332575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.350711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6602649962eb4dfdb6e43b225af1a7e1, user_id=c4cc5c910a444d41b7c8c94d6a2eea7b, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.377898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182444-108f-43bc-9024-2dc963e1cdb5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.492226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1108/2217] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:10 2022] PUT /identity/v3/projects/cb6be285203847918fec55b7564f6d28/users/41353328ca434358acf5c10c7088ce74/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:10.492226 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.548900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1110/2218] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:10 2022] PUT /identity/v3/projects/6602649962eb4dfdb6e43b225af1a7e1/users/c4cc5c910a444d41b7c8c94d6a2eea7b/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:10.548900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.548900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.549666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.593290 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a801651-88c3-49d2-98f1-43fe60769a83 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.595741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] MFA Rules not processed for user `41353328ca434358acf5c10c7088ce74`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:10.597180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:10.693603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.695733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.695733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.695733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.695733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.703051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.707588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1111/2219] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:10.708634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.721296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae7189-67aa-45b4-bccb-4ad3cd0489db None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.728658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.729469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.729976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.730550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.731583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.733552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.753150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1112/2220] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:10.753150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.761144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b30892-3461-4ce6-962d-e67c5aacef4e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.767210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.768019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.768307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.768594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.769542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.772420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598aeb7-b774-440c-9791-6dab811ed621 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.799597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1109/2221] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/projects => generated 405 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:10.800843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.867261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045fa3-9ab9-4861-a276-7139cf9968ef admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:10.873392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:10.874361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.874655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.874939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] PATH_INFO: `/v3/projects/6602649962eb4dfdb6e43b225af1a7e1/users/c4cc5c910a444d41b7c8c94d6a2eea7b/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.876964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6602649962eb4dfdb6e43b225af1a7e1, user_id=c4cc5c910a444d41b7c8c94d6a2eea7b, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:10.879650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:10.890685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1110/2222] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:10 2022] PUT /identity/v3/projects/6602649962eb4dfdb6e43b225af1a7e1/users/c4cc5c910a444d41b7c8c94d6a2eea7b/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:10.891561 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:10.925527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1113/2223] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:10.927304 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:10.927640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:10.927918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:10.968523 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] MFA Rules not processed for user `c4cc5c910a444d41b7c8c94d6a2eea7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:10.969686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:10.983762 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28951-e029-4088-9779-5f28fc5a7d70 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.054191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1114/2224] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:11.054191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.069757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.070470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.070994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.071256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.071957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.073952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.130093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b7f2-caa7-4452-9d1e-0d028fa17ae9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.138725 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.139505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.139798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.140078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.140815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.142859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.169243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1111/2225] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:10 2022] POST /identity/v3/users => generated 309 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:11.172198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.174396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:11.250361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83941b7-fc31-4305-b80c-76193aee409e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.257527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.258808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.259344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.259805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.267889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1115/2226] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:11 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:11.269063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.273018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:11.313436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c9254-fb8a-46c2-ac04-627121b289d2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.333507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.336320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.341386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.341386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.341386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.341970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.362688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.364422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.368843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1116/2227] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:11.370624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.386849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1112/2228] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:11.387842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.390235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddcf64-12d6-44b8-9ac7-44323cc6f1b6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.396808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.398753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.398753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.398753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.399702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.399884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3042-62c8-4a8b-895d-30a93890ab8d service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.404997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.405838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.406272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.406582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] PATH_INFO: `/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4/users/c047721e8b5b46118bd3c53a871e487c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.415996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.442205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1117/2229] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:11.449374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2494f4632e4bfaa6e4e0cf107b29b4, user_id=c047721e8b5b46118bd3c53a871e487c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.450486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.455921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.474068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107121a-4396-451f-97ab-7883e0588d99 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.493631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.494575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.494866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.495148 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] PATH_INFO: `/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573/users/0062c815a6b946c8908ca5baecdf55cf/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.535974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50fab6ecf8d54fb8a441cfeb36aea573, user_id=0062c815a6b946c8908ca5baecdf55cf, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.542326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1113/2230] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:11 2022] PUT /identity/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4/users/c047721e8b5b46118bd3c53a871e487c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:11.546834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.547841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.566828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:11.644319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb116f0-e81d-4b39-8ab5-68c67d9d14df admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.651351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.652126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.652620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.652732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.653485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.656045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.671712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1114/2231] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:11.672584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.683324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c448-302a-4593-bbcc-b69a132f50e6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.688795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.689705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.690032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.690319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] PATH_INFO: `/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4/users/c047721e8b5b46118bd3c53a871e487c/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.693744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2494f4632e4bfaa6e4e0cf107b29b4, user_id=c047721e8b5b46118bd3c53a871e487c, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.698084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddc7cd-fb80-441a-963e-f6f800b6270f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.701506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1118/2232] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:11 2022] PUT /identity/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573/users/0062c815a6b946c8908ca5baecdf55cf/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:11.706058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.721882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1115/2233] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:11 2022] PUT /identity/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4/users/c047721e8b5b46118bd3c53a871e487c/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:11.725875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.726180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.726472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.768054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] MFA Rules not processed for user `c047721e8b5b46118bd3c53a871e487c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:11.769305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:11.792589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e29db-497a-42b8-b57f-5f57c18d3a2e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.865628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.865868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1116/2234] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:11.865970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.865970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.865970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.865970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.865970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bed4e-10a7-4d72-870d-2eac168b410d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.879933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1119/2235] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:11.888900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:11.906140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935161e-d131-4c2e-8116-cedd6cdcebba None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:11.915439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:11.920345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.920345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.920345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] PATH_INFO: `/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573/users/0062c815a6b946c8908ca5baecdf55cf/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:11.920345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50fab6ecf8d54fb8a441cfeb36aea573, user_id=0062c815a6b946c8908ca5baecdf55cf, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:11.922438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:11.935696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1117/2236] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:11 2022] PUT /identity/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573/users/0062c815a6b946c8908ca5baecdf55cf/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:11.946729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:11.947027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:11.947355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.000459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] MFA Rules not processed for user `0062c815a6b946c8908ca5baecdf55cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:12.001893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:12.107024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1120/2237] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:12.123900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:12.201147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f5c65-0fba-4334-9bb4-2181bb1b8b09 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:12.208831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:12.209573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:12.209959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:12.210232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.314071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:12.315617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651fdc3-7dc0-425a-891c-1e6cbcb6ac3c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:12.327823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1118/2238] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:12.407319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:12.423966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e492-6c24-4ecf-8735-e4c5449323d9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:12.432730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:12.435519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:12.441635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:12.441635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.452708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:12.454435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:12.468825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1121/2239] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:12.528428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:12.528813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:12.529190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.572732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] MFA Rules not processed for user `f86340c271b64c14b30834c51a477ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:12.574089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:12.652955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1119/2240] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:12.657480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:12.658989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:12.738237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8965-4157-4511-96eb-b29a231c01a7 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:12.744686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:12.745421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:12.745716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:12.746417 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.747150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:12.749162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:12.853874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1122/2241] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:12 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:12.855446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:12.867999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa524c0-338e-47b1-bb48-fb04705e2823 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:12.884557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:12.927542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:12.928326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:12.928603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:12.928880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:12.929587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:12.931575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.032078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1120/2242] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:12 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:13.036105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.047129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c80f-bfb1-4130-932e-9e3a4c2c2386 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:13.051990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:13.052651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.052942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.053231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.054025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:13.056186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.067286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1123/2243] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:13.071537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.081511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9e82-096b-48c2-8dff-59bdba760cb6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:13.086247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:13.087129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.087413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.087685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] PATH_INFO: `/v3/projects/dae55def91d5421a92acf2b1deef5f21/users/b432d9c4986742f7b44772dafe7f384e/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.102959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae55def91d5421a92acf2b1deef5f21, user_id=b432d9c4986742f7b44772dafe7f384e, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:13.110134 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.234366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1121/2244] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:13 2022] PUT /identity/v3/projects/dae55def91d5421a92acf2b1deef5f21/users/b432d9c4986742f7b44772dafe7f384e/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:13.238612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.254059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4baf-9345-43d9-9804-f352023665f3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:13.278022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.280794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:13.344650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:13.349679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.349679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.349679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.349679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:13.349679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.370109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1124/2245] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:13.376368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.402174 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602503-1f70-40f9-abc1-1842ab57845d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:13.409967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:13.411151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.411583 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.411855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] PATH_INFO: `/v3/projects/dae55def91d5421a92acf2b1deef5f21/users/b432d9c4986742f7b44772dafe7f384e/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.417213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39156f28-7df5-4a64-be6d-12a9f33cd4c1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:13.418441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae55def91d5421a92acf2b1deef5f21, user_id=b432d9c4986742f7b44772dafe7f384e, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:13.423998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c63be8-df46-4481-ae8c-986ad5e84584 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.431804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:13.435169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.435545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.436987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] PATH_INFO: `/v3/users/20f4fcdc2f6d4761a9e0c14e85e23a9e` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.439864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f4fcdc2f6d4761a9e0c14e85e23a9e)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:13.442304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:13.446805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1125/2246] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:13 2022] PUT /identity/v3/projects/dae55def91d5421a92acf2b1deef5f21/users/b432d9c4986742f7b44772dafe7f384e/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:13.658374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f4fcdc2f6d4761a9e0c14e85e23a9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:13.673433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f4fcdc2f6d4761a9e0c14e85e23a9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:13.808731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f4fcdc2f6d4761a9e0c14e85e23a9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:13.810904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f4fcdc2f6d4761a9e0c14e85e23a9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:13.811877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f4fcdc2f6d4761a9e0c14e85e23a9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:13.833556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1122/2247] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:13 2022] DELETE /identity/v3/users/20f4fcdc2f6d4761a9e0c14e85e23a9e => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:13.838279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:13.841281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:13.843292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:13.890516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:13.892189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:13.952352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.954508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:13.974874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1126/2248] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:13.985853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:13.987645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:14.029855 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbd99-e359-4079-992c-5553e8dbfed5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:14.034671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:14.035569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:14.035860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:14.036144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] PATH_INFO: `/v3/users/41353328ca434358acf5c10c7088ce74` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:14.037386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41353328ca434358acf5c10c7088ce74)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:14.039480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:14.075673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a694d-c60f-4098-9aa9-1206b06afb73 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:14.079503 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:14.080173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:14.080464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:14.080749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:14.091416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:14.092869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda851e-d60d-4c93-a565-4ec16b8edb4a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:14.105512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1127/2249] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:14.485085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41353328ca434358acf5c10c7088ce74'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:14.505263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41353328ca434358acf5c10c7088ce74'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:14.631079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41353328ca434358acf5c10c7088ce74'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:14.631357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41353328ca434358acf5c10c7088ce74'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:14.632090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41353328ca434358acf5c10c7088ce74'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:14.660815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1123/2250] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:13 2022] DELETE /identity/v3/users/41353328ca434358acf5c10c7088ce74 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:14.662296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:14.662698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:14.662861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:14.694345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:14.694345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:14.751937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:14.753416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:14.803421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1128/2251] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:14.821761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:14.823857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:14.856070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:14.891181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162a55-5df7-4f4d-9bd9-39a9d3688050 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:14.896653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:14.897538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:14.897856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:14.898472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] PATH_INFO: `/v3/projects/06c5fb76b95243af9a6fd73f18234ed7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:14.900295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c5fb76b95243af9a6fd73f18234ed7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:14.902235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:14.941236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ced6b-89af-4a87-a516-6f782e32119c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:14.950408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:14.951139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:14.951432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:14.951722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:14.964673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:14.966489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:14.976862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1129/2252] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:15.109150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:15.110967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:15.126215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c5fb76b95243af9a6fd73f18234ed7', 'request_id': 'req-ca55bb36-a5d1-4776-ab5c-72443878403b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.139886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e683e3-efda-4931-a827-d15bcde8bc7c service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:15.157151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c5fb76b95243af9a6fd73f18234ed7', 'request_id': 'req-ca55bb36-a5d1-4776-ab5c-72443878403b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.171985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c5fb76b95243af9a6fd73f18234ed7', 'request_id': 'req-ca55bb36-a5d1-4776-ab5c-72443878403b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.308520 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] The token cache is being invalidate because project 06c5fb76b95243af9a6fd73f18234ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:15.308859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.309691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.316069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1124/2253] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:14 2022] DELETE /identity/v3/projects/06c5fb76b95243af9a6fd73f18234ed7 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:15.318083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:15.325850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:15.332690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:15.334971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:15.335279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:15.339589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] PATH_INFO: `/v3/users/059be17987d64fd7b98beacea54aa2c5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:15.343185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=059be17987d64fd7b98beacea54aa2c5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:15.346963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:15.367352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55bb36-a5d1-4776-ab5c-72443878403b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:15.374701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:15.376008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:15.376851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:15.377516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] PATH_INFO: `/v3/users/f32252adea564651b301b68033880c2a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:15.379341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32252adea564651b301b68033880c2a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:15.383908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:15.678690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059be17987d64fd7b98beacea54aa2c5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.681639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059be17987d64fd7b98beacea54aa2c5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.704530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32252adea564651b301b68033880c2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.735446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32252adea564651b301b68033880c2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.750544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059be17987d64fd7b98beacea54aa2c5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.750774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059be17987d64fd7b98beacea54aa2c5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.761568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059be17987d64fd7b98beacea54aa2c5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.785910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1130/2254] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:15 2022] DELETE /identity/v3/users/059be17987d64fd7b98beacea54aa2c5 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:15.785910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:15.787792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:15.847289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32252adea564651b301b68033880c2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.848211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32252adea564651b301b68033880c2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.848920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93110f70-3de0-4bf8-a7d9-bf1df50c4401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32252adea564651b301b68033880c2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:15.879130 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1125/2255] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:15 2022] DELETE /identity/v3/users/f32252adea564651b301b68033880c2a => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:15.883156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:15.884135 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:15.884675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:15.926648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:15.929238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:15.940198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5bd9e-7691-46ce-b0a6-938cbf88df2a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:15.998702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:16.056751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1126/2256] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:16.064717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:16.070389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:16.070980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:16.071504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] PATH_INFO: `/v3/projects/cb6be285203847918fec55b7564f6d28` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:16.073273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6be285203847918fec55b7564f6d28)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:16.076300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:16.182490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:16.191022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.243828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6be285203847918fec55b7564f6d28', 'request_id': 'req-e066e45e-c5c9-4f04-8d68-69fe57e14edb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.256293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6be285203847918fec55b7564f6d28', 'request_id': 'req-e066e45e-c5c9-4f04-8d68-69fe57e14edb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.324438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6be285203847918fec55b7564f6d28', 'request_id': 'req-e066e45e-c5c9-4f04-8d68-69fe57e14edb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.355164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] The token cache is being invalidate because project cb6be285203847918fec55b7564f6d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:16.355419 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.360239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.362604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38207c04-a50a-4670-abfe-43328e1b8147 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:16.364752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1131/2257] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:15 2022] DELETE /identity/v3/projects/cb6be285203847918fec55b7564f6d28 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:16.369514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:16.371384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.474620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:16.475681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:16.476552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:16.477023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:16.480080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.518205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e45e-c5c9-4f04-8d68-69fe57e14edb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:16.525157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:16.527900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:16.528473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:16.528976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] PATH_INFO: `/v3/users/0062c815a6b946c8908ca5baecdf55cf` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:16.531283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0062c815a6b946c8908ca5baecdf55cf)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:16.534865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:16.563813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:16.572302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:16.587526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1127/2258] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:16.588426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:16.590132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.667512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0e096-9b1f-493b-ab0e-f1b47650bbb7 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:16.673199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:16.673911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:16.674199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:16.674820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:16.676688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.747294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:16.748906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:16.760654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1128/2259] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:16.801472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:16.803058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:16.832148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f196c5-ba3d-4510-a6bb-9ab2dbb4b4e7 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:16.841107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22191079-0585-4372-a942-77cbba740bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:16.842042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191079-0585-4372-a942-77cbba740bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:16.844708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191079-0585-4372-a942-77cbba740bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:16.844708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191079-0585-4372-a942-77cbba740bca admin admin] PATH_INFO: `/v3/projects/e432951951204b1499b5d97cc2114e5c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:16.846806 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22191079-0585-4372-a942-77cbba740bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e432951951204b1499b5d97cc2114e5c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:16.848799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22191079-0585-4372-a942-77cbba740bca admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:16.851511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0062c815a6b946c8908ca5baecdf55cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.852919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0062c815a6b946c8908ca5baecdf55cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.957389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0062c815a6b946c8908ca5baecdf55cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.958200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0062c815a6b946c8908ca5baecdf55cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:16.972988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3acd4877-e36d-47ad-b5b9-5b13f3954364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0062c815a6b946c8908ca5baecdf55cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.012574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1132/2260] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:16 2022] DELETE /identity/v3/users/0062c815a6b946c8908ca5baecdf55cf => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:17.059816 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e432951951204b1499b5d97cc2114e5c', 'request_id': 'req-22191079-0585-4372-a942-77cbba740bca'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.070593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e432951951204b1499b5d97cc2114e5c', 'request_id': 'req-22191079-0585-4372-a942-77cbba740bca'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.085146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e432951951204b1499b5d97cc2114e5c', 'request_id': 'req-22191079-0585-4372-a942-77cbba740bca'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.280478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] The token cache is being invalidate because project e432951951204b1499b5d97cc2114e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:17.280886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.284236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-22191079-0585-4372-a942-77cbba740bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:17.288355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1129/2261] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:16 2022] DELETE /identity/v3/projects/e432951951204b1499b5d97cc2114e5c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:17.294472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:17.294975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:17.295352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:17.335881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:17.344183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:17.484993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1133/2262] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:17.495903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:17.496535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:17.496980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:17.525733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:17.527426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:17.544351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1130/2263] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:17.548981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:17.561897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d62e14-1d52-4750-81d9-efbef91350b8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:17.573919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:17.574611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:17.574903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:17.575188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:17.576001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:17.576322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:17.584318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:17.622992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1134/2264] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:17.624633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:17.636157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fc181-b0ee-4360-a834-dc12f9aa6480 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:17.686371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:17.687200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:17.687493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:17.687770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:17.688485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:17.705459 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:17.925916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1131/2265] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:17 2022] POST /identity/v3/projects => generated 379 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:17.930727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:17.941394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38f81-eb3b-45b0-8089-2873ccc2db39 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:17.998452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.000227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.000405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.000917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.001202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.001486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.002289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.004435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.015796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16716d-f46a-46c2-903e-ad2ea6fafa51 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.020657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.021641 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.022046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.022538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] PATH_INFO: `/v3/projects/e0b7a80ee40b42ce97debf7dcfec27f9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.023989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b7a80ee40b42ce97debf7dcfec27f9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.026133 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.139805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1135/2266] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:17 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:18.143815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.145928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.158501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca514ac1-884c-487a-9925-f8a1792a5240 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.163438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.164380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.165173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.165456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] PATH_INFO: `/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.166908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50fab6ecf8d54fb8a441cfeb36aea573)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.168864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.232446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b7a80ee40b42ce97debf7dcfec27f9', 'request_id': 'req-ef99f002-555b-443d-adc5-64398a6165da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.243470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b7a80ee40b42ce97debf7dcfec27f9', 'request_id': 'req-ef99f002-555b-443d-adc5-64398a6165da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.254526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b7a80ee40b42ce97debf7dcfec27f9', 'request_id': 'req-ef99f002-555b-443d-adc5-64398a6165da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.361370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50fab6ecf8d54fb8a441cfeb36aea573', 'request_id': 'req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.369816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50fab6ecf8d54fb8a441cfeb36aea573', 'request_id': 'req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.426815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] The token cache is being invalidate because project e0b7a80ee40b42ce97debf7dcfec27f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:18.427429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.428350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.430286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1132/2267] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:17 2022] DELETE /identity/v3/projects/e0b7a80ee40b42ce97debf7dcfec27f9 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:18.432095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.434509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.514438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f002-555b-443d-adc5-64398a6165da admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.572235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.572863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.573148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.573456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.574372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.575989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.589011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1133/2268] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:18.589011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.589011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.603910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50fab6ecf8d54fb8a441cfeb36aea573', 'request_id': 'req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.609591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb47483-2b1a-4599-8726-e75669ef1e59 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.618834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.618834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.618834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.618834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] PATH_INFO: `/v3/users/c047721e8b5b46118bd3c53a871e487c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.619228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c047721e8b5b46118bd3c53a871e487c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.620209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.623314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] The token cache is being invalidate because project 50fab6ecf8d54fb8a441cfeb36aea573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:18.623896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.625020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:18.631308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1136/2269] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:18 2022] DELETE /identity/v3/projects/50fab6ecf8d54fb8a441cfeb36aea573 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:18.634171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.648548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.665622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3d9b3-8436-494e-a0c9-e06fd13bb1c0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.673452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.674214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.674590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.674874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.675630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.678141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ba55c-12b2-4521-ba3c-2b33c2a60a74 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.695243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1137/2270] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:18.695243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.695243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.695243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.716512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:18.718149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:18.730175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1138/2271] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:18.730175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:18.738676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f522c-ddb2-47bd-805c-e31559667489 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:18.745984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:18.746972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.747262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.748209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] PATH_INFO: `/v3/projects/1f05ec1c129e43ac91aa1bcf2cc7fd15/users/f8a5e44ad1ff41d5ad372f17cd4e3de2/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.770046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f05ec1c129e43ac91aa1bcf2cc7fd15, user_id=f8a5e44ad1ff41d5ad372f17cd4e3de2, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:18.773120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e387d2-464b-4e5f-aada-f9d75266c0bf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:18.905813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1139/2272] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:18 2022] PUT /identity/v3/projects/1f05ec1c129e43ac91aa1bcf2cc7fd15/users/f8a5e44ad1ff41d5ad372f17cd4e3de2/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:18.908255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d541c3-2a1c-4656-87b6-d8fd83f153b8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:18.909344 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d541c3-2a1c-4656-87b6-d8fd83f153b8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:18.909914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d541c3-2a1c-4656-87b6-d8fd83f153b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:18.940715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a3d541c3-2a1c-4656-87b6-d8fd83f153b8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:18.942331 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3d541c3-2a1c-4656-87b6-d8fd83f153b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:19.018152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c047721e8b5b46118bd3c53a871e487c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:19.028261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1140/2273] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:19.031649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:19.031649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:19.031649 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:19.040143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c047721e8b5b46118bd3c53a871e487c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:19.057522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:19.084724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] MFA Rules not processed for user `f8a5e44ad1ff41d5ad372f17cd4e3de2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:19.084724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:19.146622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c047721e8b5b46118bd3c53a871e487c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:19.147506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c047721e8b5b46118bd3c53a871e487c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:19.148603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c047721e8b5b46118bd3c53a871e487c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:19.167466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1134/2274] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:18 2022] DELETE /identity/v3/users/c047721e8b5b46118bd3c53a871e487c => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:19.178240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1141/2275] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:19.179360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:19.181562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:19.188719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:19.202546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce73c-ada0-43be-99d6-fa398a2f10c8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:19.203183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:19.204834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:19.347813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a350c-98e0-473d-82d3-e258eb78e98a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:19.352284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:19.354635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:19.356539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:19.356824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:19.373366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:19.374737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:19.375070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:19.376736 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:19.377857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:19.378151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:19.381050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:19.382510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:19.394146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:19.421054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1142/2276] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:19.422449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:19.440421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e716c3-1032-4796-8d93-1f7708ffdf17 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:19.445487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:19.446909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:19.447251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:19.447525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:19.448762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:19.450377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.080957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1135/2277] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:19 2022] POST /identity/v3/projects => generated 377 bytes in 877 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:20.082927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.085640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1143/2278] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:19 2022] POST /identity/v3/projects => generated 375 bytes in 663 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:20.089681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.158298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc56c3-1373-45cb-a905-86f62adb7622 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.174973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98c78d-b542-4d3c-abce-3a82419443f0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.180164 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.180849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.181144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.181431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.182226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.184184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.218967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.219648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.219930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.220205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.220900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.222873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.448329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1136/2279] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:20 2022] POST /identity/v3/users => generated 289 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:20.453341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1144/2280] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:20 2022] POST /identity/v3/users => generated 288 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:20.455918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.457928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.469473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215c5e3-f503-4af2-a30b-afcd9d5726e8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.469951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba86fb-214e-4a46-8a7c-a97a6cc14384 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.474737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.475399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.475686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.475972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.476349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.476713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.477449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.478013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.478431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.478738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.479665 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.481716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.489451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1137/2281] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:20.496399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.499188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:20.502303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1145/2282] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:20.503514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.513557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1484ae-67d9-499b-a9ef-586eaa2072df admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.514650 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a508e4-afdc-434e-a0f7-c3444ac4f257 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.519055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.520109 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.520463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.520750 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] PATH_INFO: `/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.521337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.521996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2494f4632e4bfaa6e4e0cf107b29b4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.522981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.523499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.524575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.527848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.528786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.530883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.551659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1146/2283] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:20.554714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.567762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908e9f-7e4f-4662-a6d7-b8c1789f0eed admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.575052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.576377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.576890 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.577243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.578177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.580080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:20.596072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1147/2284] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:20.596926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:20.607641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e79ad2-7916-4c3e-83ab-be6a61cb89b2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:20.612469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:20.613376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:20.613668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:20.622513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] PATH_INFO: `/v3/projects/aa7aa5a39ff6403ebeab9c82106cead4/users/b8b8f6b8e5144ff686330603848f68c8/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:20.631405 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7aa5a39ff6403ebeab9c82106cead4, user_id=b8b8f6b8e5144ff686330603848f68c8, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:20.634959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:21.112461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1148/2285] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:20 2022] PUT /identity/v3/projects/aa7aa5a39ff6403ebeab9c82106cead4/users/b8b8f6b8e5144ff686330603848f68c8/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:21.113298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:21.148840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2494f4632e4bfaa6e4e0cf107b29b4', 'request_id': 'req-eb79585f-27f2-45dd-b14a-1827d317ca8a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:21.158780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2494f4632e4bfaa6e4e0cf107b29b4', 'request_id': 'req-eb79585f-27f2-45dd-b14a-1827d317ca8a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:21.166771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2494f4632e4bfaa6e4e0cf107b29b4', 'request_id': 'req-eb79585f-27f2-45dd-b14a-1827d317ca8a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:21.204345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04acd71-fffc-48d9-acc4-944805b5d14b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:21.271631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:21.272627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.273063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.273355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] PATH_INFO: `/v3/projects/7bf98033e0e548d4b9977c0ee78a80cb/users/65b11b36fc81420a89aa5d650a089a40/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.293234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf98033e0e548d4b9977c0ee78a80cb, user_id=65b11b36fc81420a89aa5d650a089a40, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:21.296514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:21.319859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] The token cache is being invalidate because project fe2494f4632e4bfaa6e4e0cf107b29b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:21.320284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:21.321468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-eb79585f-27f2-45dd-b14a-1827d317ca8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:21.327737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1138/2286] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:20 2022] DELETE /identity/v3/projects/fe2494f4632e4bfaa6e4e0cf107b29b4 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:21.329292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96edf3-6990-4e75-b162-a88421d023ee None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.329586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96edf3-6990-4e75-b162-a88421d023ee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.329899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96edf3-6990-4e75-b162-a88421d023ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.362405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5c96edf3-6990-4e75-b162-a88421d023ee None None] MFA Rules not processed for user `b8b8f6b8e5144ff686330603848f68c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:21.363650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c96edf3-6990-4e75-b162-a88421d023ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.451956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1139/2287] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:21.458326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce90efa-a2a6-489d-89c4-e0b07d13a5a6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.458760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce90efa-a2a6-489d-89c4-e0b07d13a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.459311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce90efa-a2a6-489d-89c4-e0b07d13a5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.463765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1149/2288] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:21 2022] PUT /identity/v3/projects/7bf98033e0e548d4b9977c0ee78a80cb/users/65b11b36fc81420a89aa5d650a089a40/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:21.464787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:21.471802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.491469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-cce90efa-a2a6-489d-89c4-e0b07d13a5a6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:21.492918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cce90efa-a2a6-489d-89c4-e0b07d13a5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.568235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f1852-07f5-4dd5-a6c7-b04d415b21eb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:21.571116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1140/2289] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:21.573140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.573513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:21.574191 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.574321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.574804 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.574921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.575129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.581229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:21.583478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa527aa-5ecf-43c0-944d-dc26f7eb5163 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:21.621951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] MFA Rules not processed for user `65b11b36fc81420a89aa5d650a089a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:21.628468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.712751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1141/2290] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:21.713967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:21.717436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.778163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1150/2291] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:21 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:21.779833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.780186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.780998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.812656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:21.812850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9b1e7-c072-42b8-b07e-1541448d0a4f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:21.815673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.923565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:21.924234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:21.924521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:21.924871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:21.935617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.964281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1151/2292] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:21.964281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:21.967870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:21.996694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61ed70-1a61-4f25-9a74-070776649af6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.014593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.015835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.016219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.016506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.061115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.063443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.083473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1142/2293] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:22.084723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.099681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84ff4-0d99-4b7e-b157-cf3fc14cabca service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.107547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.108439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.108873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.109158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.109898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.112259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce3592-d4ab-4702-98e1-7c2853a294f2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.140307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.142154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.159635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1152/2294] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:22.160350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.176979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff121bd2-2f14-4700-9060-5587938ee9da service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.185267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.186198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.186601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.186953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.187801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:22.188220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.190057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.208380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1153/2295] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:22.213635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.225007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d7d03-1fc9-4390-a800-45f1c56996ea admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.231372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.232210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.232636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.233067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.233911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.236284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.310445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1143/2296] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:22 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:22.311937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.312248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.312623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.353253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] MFA Rules not processed for user `b432d9c4986742f7b44772dafe7f384e`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:22.354601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:22.410741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1154/2297] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:22 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:22.411695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.442876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1144/2298] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:22.453586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.472281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daec81-e156-44b3-be98-ce023a8142a4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.530547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85670b8-36b2-4db5-85df-2e4f557b8833 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.538024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.538809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.539107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.539484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.540801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.542486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.547883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.549166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.549524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.549944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.551023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.553103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.557051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1155/2299] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:22.557051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.557555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:22.644501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374c9b6-3c9f-4d1f-aad0-124c356e8181 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.650807 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.651478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.651770 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.652054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.653969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:22.689836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1145/2300] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:22 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:22.690711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.701288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f29920-e14f-46cf-8c22-07fad6a4bbce admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.706397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.707418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.707742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.708089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] PATH_INFO: `/v3/projects/0c85e4ba7507467482af03e6e03a0b16/users/d38ad0ad49dd47b78198ffde489cc307/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.718502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.720346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.731682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c85e4ba7507467482af03e6e03a0b16, user_id=d38ad0ad49dd47b78198ffde489cc307, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.735175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.737978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1156/2301] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:22.739893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.752552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e46785-a9b9-4ae7-9344-c7bec38a26ea service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.757677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.758959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.759254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.759534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.760353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.762119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.782338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1157/2302] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:22.786626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.798299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf37e-db5c-43a5-ad93-0a58ad245f4d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.804321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.805029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.805322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.805634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.806840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.808387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.823255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1158/2303] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:22.824334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.834170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e64b3-f98d-4000-b1e4-49664385cf31 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.839690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.840684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.840976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.841263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] PATH_INFO: `/v3/projects/24503a58fe7a45f2a81bec1d186145de/users/b85f309df4244118af50ee38aeb4c9f7/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.856862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=24503a58fe7a45f2a81bec1d186145de, user_id=b85f309df4244118af50ee38aeb4c9f7, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.859586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4649e3c-c1b5-4bbb-b784-57ad97d676ab admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:22.913449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1146/2304] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:22 2022] PUT /identity/v3/projects/0c85e4ba7507467482af03e6e03a0b16/users/d38ad0ad49dd47b78198ffde489cc307/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:22.917438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:22.979453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763b4cd-88f4-4e56-9707-a64fcad9b554 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:22.985744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:22.986555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:22.986780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:22.987064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:22.987815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:22.989428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.000791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1147/2305] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:23.004891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.016165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1159/2306] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:22 2022] PUT /identity/v3/projects/24503a58fe7a45f2a81bec1d186145de/users/b85f309df4244118af50ee38aeb4c9f7/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:23.020189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3579e-58f8-41f4-98fa-6d5d0b5ac15b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.029163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.029542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.029997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.057019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] MFA Rules not processed for user `b85f309df4244118af50ee38aeb4c9f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:23.058742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:23.084491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.085392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.085675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.086003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] PATH_INFO: `/v3/projects/0c85e4ba7507467482af03e6e03a0b16/users/d38ad0ad49dd47b78198ffde489cc307/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.088566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c85e4ba7507467482af03e6e03a0b16, user_id=d38ad0ad49dd47b78198ffde489cc307, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.091243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643a9e1-3b19-42e4-beff-09f076b7186c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.101944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1148/2307] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:23 2022] PUT /identity/v3/projects/0c85e4ba7507467482af03e6e03a0b16/users/d38ad0ad49dd47b78198ffde489cc307/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:23.112390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.112760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.113042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.126235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1160/2308] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:23.131267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.142420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] MFA Rules not processed for user `d38ad0ad49dd47b78198ffde489cc307`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:23.143594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:23.145946 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600be7-6d59-4061-bc18-fbd481013e82 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.222413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.223446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.223824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.224188 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.224981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.227361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.254061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1149/2309] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:23.409598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1161/2310] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:23 2022] POST /identity/v3/projects => generated 369 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:23.413602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.425972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b768b-d7e9-47a0-8ff6-8f1b7f64728c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.517333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.517333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.517333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.517333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.517705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.517705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.575742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:23.653757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.675891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1150/2311] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:23 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:23.685549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.695949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2791f6-6f6b-4233-a994-b32864a23684 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.701053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.701955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.702346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.702713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.703527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.705274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.721975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1151/2312] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:23.728445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.733824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee37a5-ea36-4439-9a93-9836ddad2602 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.743277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.743865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.744221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.744438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.747078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8dc2-c608-42b7-9d8f-1c1f5b901509 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:23.758887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:23.759799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:23.760083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:23.760361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] PATH_INFO: `/v3/projects/590af2bf3f294e4597ca554de2112482/users/0ce45f7d78ea49848d7d56fb62127160/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:23.801660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=590af2bf3f294e4597ca554de2112482, user_id=0ce45f7d78ea49848d7d56fb62127160, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.804387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.837490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:23.838937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:23.850251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1162/2313] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:23.932809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1152/2314] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:23 2022] PUT /identity/v3/projects/590af2bf3f294e4597ca554de2112482/users/0ce45f7d78ea49848d7d56fb62127160/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:23.936697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:23.950051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942877fd-1fe3-47bb-a7bc-62fa967e0955 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:24.022505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:24.023297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.033827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.033827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.033827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:24.033827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487b952-bfd0-402b-aa05-429eb02f6e36 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:24.042079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1163/2315] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:24.043596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:24.054043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3ba7-a292-49e4-bade-736e6da9fe6d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:24.069955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:24.071697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.072084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.072489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] PATH_INFO: `/v3/projects/590af2bf3f294e4597ca554de2112482/users/0ce45f7d78ea49848d7d56fb62127160/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.074712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=590af2bf3f294e4597ca554de2112482, user_id=0ce45f7d78ea49848d7d56fb62127160, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:24.084470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:24.096735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1153/2316] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:24 2022] PUT /identity/v3/projects/590af2bf3f294e4597ca554de2112482/users/0ce45f7d78ea49848d7d56fb62127160/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:24.109035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.109332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.110265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.142623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:24.152972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] MFA Rules not processed for user `0ce45f7d78ea49848d7d56fb62127160`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:24.156047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:24.221042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccec3-b09c-4adb-b2a1-c90b7d9ac9b2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:24.229083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:24.229781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.230091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.230430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.232628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:24.251833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1164/2317] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:24.299578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:24.301304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:24.311470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1154/2318] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:24.404070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:24.422104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cd9bf-fe6e-4233-ad7d-1076085490db None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:24.425968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:24.426724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.427011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.427317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.438512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:24.441156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5b988-ea6e-4d9b-b917-82d8d38bae97 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:24.452519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1165/2319] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:24.777468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:24.788247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb404e5d-0a50-4731-bcbc-3bbdabf7e93c service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:24.791902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:24.792755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:24.793040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:24.793323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:24.848512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:24.862731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:24.864436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:24.875739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1155/2320] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:26.253394 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:27.177185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb8e0-99b0-4e91-8165-d0fe4a1b1bda None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:27.177185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb8e0-99b0-4e91-8165-d0fe4a1b1bda None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:27.177185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb8e0-99b0-4e91-8165-d0fe4a1b1bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:27.202323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6d5cb8e0-99b0-4e91-8165-d0fe4a1b1bda None None] MFA Rules not processed for user `f86340c271b64c14b30834c51a477ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:27.202323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6d5cb8e0-99b0-4e91-8165-d0fe4a1b1bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:27.307091 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1166/2321] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:27.780948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:30.348505 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:30.351476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:30.445122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33c68b-1dfc-4496-9758-7fcc745c393f service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:30.450588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:30.451685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:30.452071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:30.452453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] PATH_INFO: `/v3/users/78c0cde810534d29b999ba8ef5dc5adc` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:30.454011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c0cde810534d29b999ba8ef5dc5adc)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:30.456169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:30.745217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c0cde810534d29b999ba8ef5dc5adc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:30.760868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c0cde810534d29b999ba8ef5dc5adc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:30.872277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c0cde810534d29b999ba8ef5dc5adc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:30.873388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c0cde810534d29b999ba8ef5dc5adc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:30.874069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c0cde810534d29b999ba8ef5dc5adc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:30.890630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1156/2322] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:30 2022] DELETE /identity/v3/users/78c0cde810534d29b999ba8ef5dc5adc => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:30.901267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:30.902722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:30.902910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:30.924624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:30.926057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:30.963581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:30.987040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1167/2323] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:30.997356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:30.998807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:31.064389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103296f-f08e-4136-afac-9e9a7e5dda4b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:31.071673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:31.072821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:31.073302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:31.074416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:31.085727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:31.087469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:31.093259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:31.098901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1157/2324] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:31.563082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:31.566484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:31.578893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fa450-986d-46ab-87c4-f8dbf00451b6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:31.583734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:31.584813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:31.585243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:31.585673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] PATH_INFO: `/v3/projects/475c10fd2c5e4245965e4d915b0cc4f2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:31.587719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=475c10fd2c5e4245965e4d915b0cc4f2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:31.589694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:31.730409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475c10fd2c5e4245965e4d915b0cc4f2', 'request_id': 'req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:31.737240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475c10fd2c5e4245965e4d915b0cc4f2', 'request_id': 'req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:31.850167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475c10fd2c5e4245965e4d915b0cc4f2', 'request_id': 'req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:31.862742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] The token cache is being invalidate because project 475c10fd2c5e4245965e4d915b0cc4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:31.863038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:31.863701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:31.865369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1168/2325] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:31 2022] DELETE /identity/v3/projects/475c10fd2c5e4245965e4d915b0cc4f2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:31.869952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:31.879812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f9fad-3f05-43b5-9e06-21e070c2f5a1 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:31.945403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:31.946331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:31.946627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:31.946908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] PATH_INFO: `/v3/users/08c2c1384c364535bcd3828c3b242af5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:31.948126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c2c1384c364535bcd3828c3b242af5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:31.950959 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:32.172651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c2c1384c364535bcd3828c3b242af5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:32.185331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c2c1384c364535bcd3828c3b242af5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:32.328861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c2c1384c364535bcd3828c3b242af5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:32.329452 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c2c1384c364535bcd3828c3b242af5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:32.330076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c2c1384c364535bcd3828c3b242af5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:32.343223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1158/2326] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:31 2022] DELETE /identity/v3/users/08c2c1384c364535bcd3828c3b242af5 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:32.814607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:32.815864 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:32.816104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:32.818041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:32.832628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c37fb-e8a2-4799-a7a6-d4d34297d224 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:32.899678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef0851-fe36-4c9a-b061-bd4df1005502 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:32.912966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:32.912966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:32.912966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:32.912966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] PATH_INFO: `/v3/projects/aab79e44d3194af6aea35631974dc515` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:32.913985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab79e44d3194af6aea35631974dc515)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:32.913985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:32.930033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:32.930998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:32.931288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:32.931570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] PATH_INFO: `/v3/users/b85f309df4244118af50ee38aeb4c9f7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:32.932785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85f309df4244118af50ee38aeb4c9f7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:32.934876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:33.127361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab79e44d3194af6aea35631974dc515', 'request_id': 'req-fe72b8bc-a01d-47b4-b311-591dcb845a0f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.138630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab79e44d3194af6aea35631974dc515', 'request_id': 'req-fe72b8bc-a01d-47b4-b311-591dcb845a0f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.167443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab79e44d3194af6aea35631974dc515', 'request_id': 'req-fe72b8bc-a01d-47b4-b311-591dcb845a0f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.394412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] The token cache is being invalidate because project aab79e44d3194af6aea35631974dc515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:33.394623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.395565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.397461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1159/2327] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:32 2022] DELETE /identity/v3/projects/aab79e44d3194af6aea35631974dc515 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:33.401972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:33.403350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:33.406903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85f309df4244118af50ee38aeb4c9f7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.408017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85f309df4244118af50ee38aeb4c9f7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.468199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72b8bc-a01d-47b4-b311-591dcb845a0f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:33.474092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:33.474982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:33.475268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:33.475548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] PATH_INFO: `/v3/users/c4cc5c910a444d41b7c8c94d6a2eea7b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:33.477644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4cc5c910a444d41b7c8c94d6a2eea7b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:33.479967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:33.518556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85f309df4244118af50ee38aeb4c9f7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.519289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85f309df4244118af50ee38aeb4c9f7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.532920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-892ba093-3f91-4d99-a13e-06ece342fc9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85f309df4244118af50ee38aeb4c9f7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.553630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1169/2328] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:32 2022] DELETE /identity/v3/users/b85f309df4244118af50ee38aeb4c9f7 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:33.558533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:33.568879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:33.568879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:33.578290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:33.579528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:33.638283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1170/2329] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:33.648367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:33.649751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:33.708312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bac9a3-0b02-458a-b8ff-89cbdabe9f84 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:33.712085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:33.712740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:33.713030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:33.713315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:33.723360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:33.724819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:33.734977 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1171/2330] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:33.790819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4cc5c910a444d41b7c8c94d6a2eea7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.803805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4cc5c910a444d41b7c8c94d6a2eea7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.900562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4cc5c910a444d41b7c8c94d6a2eea7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.901183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4cc5c910a444d41b7c8c94d6a2eea7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.901856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4cc5c910a444d41b7c8c94d6a2eea7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:33.920967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1160/2331] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:33 2022] DELETE /identity/v3/users/c4cc5c910a444d41b7c8c94d6a2eea7b => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:34.429209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:34.433460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:34.462143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1c91d-836a-4c7d-a7d5-e6e06344f85f service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:34.531361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:34.532263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:34.532692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:34.532987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] PATH_INFO: `/v3/projects/24503a58fe7a45f2a81bec1d186145de` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:34.534235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24503a58fe7a45f2a81bec1d186145de)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:34.536309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:34.565324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:34.566902 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:34.581718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cca480-38a7-4a67-991d-a2b757c10ecd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:34.587284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:34.588055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:34.588346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:34.588613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] PATH_INFO: `/v3/users/f8a5e44ad1ff41d5ad372f17cd4e3de2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:34.589848 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a5e44ad1ff41d5ad372f17cd4e3de2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:34.592369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:34.595825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:34.689121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24503a58fe7a45f2a81bec1d186145de', 'request_id': 'req-d57ca445-8e71-4f53-82a8-69c2979b5772'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.695961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24503a58fe7a45f2a81bec1d186145de', 'request_id': 'req-d57ca445-8e71-4f53-82a8-69c2979b5772'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.706774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:34.818777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24503a58fe7a45f2a81bec1d186145de', 'request_id': 'req-d57ca445-8e71-4f53-82a8-69c2979b5772'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.831117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] The token cache is being invalidate because project 24503a58fe7a45f2a81bec1d186145de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:34.831484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.832129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.833699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1172/2332] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:34 2022] DELETE /identity/v3/projects/24503a58fe7a45f2a81bec1d186145de => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:34.834641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:34.837257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:34.910634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a5e44ad1ff41d5ad372f17cd4e3de2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.937276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a5e44ad1ff41d5ad372f17cd4e3de2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:34.950049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ca445-8e71-4f53-82a8-69c2979b5772 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:34.954299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:34.955187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:34.955481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:34.955767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] PATH_INFO: `/v3/projects/6602649962eb4dfdb6e43b225af1a7e1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:34.957558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6602649962eb4dfdb6e43b225af1a7e1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:34.960001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:35.089200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a5e44ad1ff41d5ad372f17cd4e3de2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.090014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a5e44ad1ff41d5ad372f17cd4e3de2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.090685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a5e44ad1ff41d5ad372f17cd4e3de2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.113681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1161/2333] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:34 2022] DELETE /identity/v3/users/f8a5e44ad1ff41d5ad372f17cd4e3de2 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:35.114691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:35.118313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.128430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6602649962eb4dfdb6e43b225af1a7e1', 'request_id': 'req-74996c29-6b0d-4c43-a58e-5787fd3bf005'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.137783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6602649962eb4dfdb6e43b225af1a7e1', 'request_id': 'req-74996c29-6b0d-4c43-a58e-5787fd3bf005'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.199264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8433fb-3d6e-4322-a397-da3d765ed7ab admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:35.204187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:35.205068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.205362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.205651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] PATH_INFO: `/v3/users/0ce45f7d78ea49848d7d56fb62127160` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.206910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ce45f7d78ea49848d7d56fb62127160)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:35.210306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:35.261556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6602649962eb4dfdb6e43b225af1a7e1', 'request_id': 'req-74996c29-6b0d-4c43-a58e-5787fd3bf005'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.279850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] The token cache is being invalidate because project 6602649962eb4dfdb6e43b225af1a7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:35.280215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.280964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74996c29-6b0d-4c43-a58e-5787fd3bf005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.285961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1173/2334] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:34 2022] DELETE /identity/v3/projects/6602649962eb4dfdb6e43b225af1a7e1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:35.285961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c031-3ea7-4411-af86-c274cad9f3cf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.285961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c031-3ea7-4411-af86-c274cad9f3cf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.285961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c031-3ea7-4411-af86-c274cad9f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.306992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2094c031-3ea7-4411-af86-c274cad9f3cf None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:35.308757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2094c031-3ea7-4411-af86-c274cad9f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.324923 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1174/2335] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:35.327224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.327647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.328008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.351073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:35.352593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.362178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1175/2336] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:35.363216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:35.364609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.448009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d486-08ca-4523-ac63-a657b5c0e78d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:35.453399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:35.454879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.455327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.455930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.468440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:35.470013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3547-5ece-4573-9fa9-f12e8ebd2a20 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:35.498477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1176/2337] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:35.498477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.498477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.498477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.501063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ce45f7d78ea49848d7d56fb62127160'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.521874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ce45f7d78ea49848d7d56fb62127160'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.538620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:35.540002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.551999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1177/2338] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:35.556397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:35.569460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b501-2243-49f5-98a3-b4669f000cec None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:35.576273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:35.577017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.577442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.577793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.579121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:35.580843 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:35.582679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:35.616741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1178/2339] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:35.616741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:35.626867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaf8b2-7aba-4532-b511-e82bfe443e41 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:35.632468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:35.633216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:35.633502 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:35.633816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:35.634528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:35.636762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:35.644617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ce45f7d78ea49848d7d56fb62127160'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.645529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ce45f7d78ea49848d7d56fb62127160'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.646717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ce45f7d78ea49848d7d56fb62127160'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:35.675501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1162/2340] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:35 2022] DELETE /identity/v3/users/0ce45f7d78ea49848d7d56fb62127160 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:35.865535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1179/2341] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:35 2022] POST /identity/v3/projects => generated 369 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:35.871565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:35.874668 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:35.965508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502b268-a8eb-4b69-8d15-3fad3c804ef9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.111234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.112282 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.112881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.113427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.114381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.134268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.192996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.195253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:36.208049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070822dd-0359-4153-8ac1-eb8211d77611 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.213223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.214237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.214607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.214969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] PATH_INFO: `/v3/projects/1f05ec1c129e43ac91aa1bcf2cc7fd15` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.216215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f05ec1c129e43ac91aa1bcf2cc7fd15)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.218212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.365000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1163/2342] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:35 2022] POST /identity/v3/users => generated 285 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:36.370615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.383103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d2fbb-4bb5-440e-86d5-544bdf9c55e1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.388538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.389227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.389513 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.389822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.390675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.392565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.403078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1164/2343] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:36.425073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.437202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057aa0-e319-40df-aa06-f8a0895ad686 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.441976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.442648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.442932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.443207 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.443931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.445935 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.467088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1165/2344] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:36.476099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.478993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:36.487049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f05ec1c129e43ac91aa1bcf2cc7fd15', 'request_id': 'req-54d56ce9-8847-42b5-be81-0a3b9c1985ba'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.495747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f05ec1c129e43ac91aa1bcf2cc7fd15', 'request_id': 'req-54d56ce9-8847-42b5-be81-0a3b9c1985ba'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.501392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48138874-8bc3-415c-aaff-4918d5aaf7ab admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.574179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.586707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.586707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.586707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] PATH_INFO: `/v3/projects/590af2bf3f294e4597ca554de2112482` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.586707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=590af2bf3f294e4597ca554de2112482)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.586707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.609980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f05ec1c129e43ac91aa1bcf2cc7fd15', 'request_id': 'req-54d56ce9-8847-42b5-be81-0a3b9c1985ba'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.622525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] The token cache is being invalidate because project 1f05ec1c129e43ac91aa1bcf2cc7fd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:36.632361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.632361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.632361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1180/2345] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:36 2022] DELETE /identity/v3/projects/1f05ec1c129e43ac91aa1bcf2cc7fd15 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:36.632361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.632361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:36.640298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56ce9-8847-42b5-be81-0a3b9c1985ba admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.644995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.645925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.646216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.646498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] PATH_INFO: `/v3/projects/5257e4d00029401298010cf0637bbcc8/users/edf5a1f6e3584004b72af3b40c0b4e2d/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.666566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5257e4d00029401298010cf0637bbcc8, user_id=edf5a1f6e3584004b72af3b40c0b4e2d, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.670195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:36.831944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1181/2346] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:36 2022] PUT /identity/v3/projects/5257e4d00029401298010cf0637bbcc8/users/edf5a1f6e3584004b72af3b40c0b4e2d/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:36.832788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:36.834407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:36.854129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590af2bf3f294e4597ca554de2112482', 'request_id': 'req-48c15d81-70ac-4df5-bf24-23e8100f2c60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.861595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590af2bf3f294e4597ca554de2112482', 'request_id': 'req-48c15d81-70ac-4df5-bf24-23e8100f2c60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.875103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590af2bf3f294e4597ca554de2112482', 'request_id': 'req-48c15d81-70ac-4df5-bf24-23e8100f2c60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:36.916225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c835-c930-4d87-a799-ccff0a7c7737 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:36.976454 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:36.977433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:36.977726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:36.978046 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] PATH_INFO: `/v3/users/b8b8f6b8e5144ff686330603848f68c8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:36.979410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b8f6b8e5144ff686330603848f68c8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:36.981635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:37.016145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] The token cache is being invalidate because project 590af2bf3f294e4597ca554de2112482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-48c15d81-70ac-4df5-bf24-23e8100f2c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1166/2347] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:36 2022] DELETE /identity/v3/projects/590af2bf3f294e4597ca554de2112482 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade177-c521-443e-b594-6da50409bec2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade177-c521-443e-b594-6da50409bec2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.029518 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade177-c521-443e-b594-6da50409bec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.058874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-93ade177-c521-443e-b594-6da50409bec2 None None] MFA Rules not processed for user `edf5a1f6e3584004b72af3b40c0b4e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:37.060269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-93ade177-c521-443e-b594-6da50409bec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:37.134815 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1167/2348] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:37.136660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.143713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.143713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.157761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:37.159063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:37.173840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1168/2349] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:37.174981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:37.176521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:37.219474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b8f6b8e5144ff686330603848f68c8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.220483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b8f6b8e5144ff686330603848f68c8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.283915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79793a30-78cb-49fb-bd5a-ea95b258989d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:37.292193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:37.292917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.293199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.293482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.295537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:37.307809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:37.309257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe940d7e-5d7e-4c24-ab3a-fd042c5ee5d7 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:37.318421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1169/2350] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:37.319765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.320074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.321560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.348942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:37.348942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:37.364795 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1170/2351] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:37.369529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:37.384841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d491ae-5e4f-43ef-a249-992ed870ba4c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:37.385036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b8f6b8e5144ff686330603848f68c8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.385036 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b8f6b8e5144ff686330603848f68c8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.387160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:37.387898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.388250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.388593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.389395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:37.389787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:37.391369 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b8f6b8e5144ff686330603848f68c8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:37.395431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:37.408485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1171/2352] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:37.413817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:37.430579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd245-a701-4e83-93b4-639fb9217430 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:37.437683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1182/2353] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:36 2022] DELETE /identity/v3/users/b8b8f6b8e5144ff686330603848f68c8 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:37.528068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:37.528744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.529176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.529539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.530262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:37.532189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:37.548294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:37.680818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1172/2354] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:37 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:37.685847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:37.698270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc77a5-bf57-4b27-a9cd-748da7992a73 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:37.764659 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:37.765348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.765663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.765947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.766935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:37.769013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:37.966806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1183/2355] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:37 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:37.969993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:37.980995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d059c3d-221a-4c25-a9d3-d6ed86147c7e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:37.985700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:37.986416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:37.986704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:37.986979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:37.987707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:37.989377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.004350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1173/2356] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:38.005815 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.016489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f1fc3-25f4-465b-a9cd-61cb317008a3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.021176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.021896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.022182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.022468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.023269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.025119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.036096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1184/2357] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:38.040154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.050881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71437863-5a60-472f-b635-c9b75d35449a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.056086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.056972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.057284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.057567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] PATH_INFO: `/v3/projects/de39d044f9cc47ddb76740368b3f1e74/users/b22dad1e58e64db5a33b8eeaa97e1032/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.087909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de39d044f9cc47ddb76740368b3f1e74, user_id=b22dad1e58e64db5a33b8eeaa97e1032, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.089564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.090768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2af95e-5c90-4ad3-a0a7-c4ad1e2e329d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.093450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:38.113427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a0054-3941-4a8b-8a95-5a7d3619b32f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.121060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.121987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.127423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.127423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] PATH_INFO: `/v3/projects/aa7aa5a39ff6403ebeab9c82106cead4` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.127423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7aa5a39ff6403ebeab9c82106cead4)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.127423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.204879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1174/2358] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:38 2022] PUT /identity/v3/projects/de39d044f9cc47ddb76740368b3f1e74/users/b22dad1e58e64db5a33b8eeaa97e1032/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:38.215599 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.215841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.216103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.247744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] MFA Rules not processed for user `b22dad1e58e64db5a33b8eeaa97e1032`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:38.248005 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:38.256277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7aa5a39ff6403ebeab9c82106cead4', 'request_id': 'req-bcab69fe-a782-4602-9a25-b56a4a4cfc58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.263940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7aa5a39ff6403ebeab9c82106cead4', 'request_id': 'req-bcab69fe-a782-4602-9a25-b56a4a4cfc58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.312195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1175/2359] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:38.316533 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.364310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7aa5a39ff6403ebeab9c82106cead4', 'request_id': 'req-bcab69fe-a782-4602-9a25-b56a4a4cfc58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.380067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] The token cache is being invalidate because project aa7aa5a39ff6403ebeab9c82106cead4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:38.380067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.387438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423fc58-173f-406c-ac7a-8566095bc8ee None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.387981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.392404 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1185/2360] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:38 2022] DELETE /identity/v3/projects/aa7aa5a39ff6403ebeab9c82106cead4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:38.393492 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.395319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:38.396672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.397355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.397646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.398004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.398716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.404010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.412381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab69fe-a782-4602-9a25-b56a4a4cfc58 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.418527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.427027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.427027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.427027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] PATH_INFO: `/v3/users/d38ad0ad49dd47b78198ffde489cc307` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.427027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38ad0ad49dd47b78198ffde489cc307)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.427027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.555942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:38.561359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1176/2361] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:38 2022] POST /identity/v3/projects => generated 383 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:38.565291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.568442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:38.679171 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38ad0ad49dd47b78198ffde489cc307'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.681397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38ad0ad49dd47b78198ffde489cc307'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.683563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcad7-0238-4379-86e2-2e699cdcb126 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.692631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.693699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.694245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.694673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.695651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.698221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.767430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38ad0ad49dd47b78198ffde489cc307'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.769142 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38ad0ad49dd47b78198ffde489cc307'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.778712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38ad0ad49dd47b78198ffde489cc307'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:38.804752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1186/2362] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:38 2022] DELETE /identity/v3/users/d38ad0ad49dd47b78198ffde489cc307 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:38.839443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1177/2363] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:38 2022] POST /identity/v3/users => generated 298 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:38.844266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.846185 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:38.919613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335274-9ad8-4e58-ad19-9aae53301cb3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.925144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.926101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.926584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.926935 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.927767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:38.929442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:38.940476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1187/2364] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:38.945129 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:38.957395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1f80e-c806-408e-872d-4f11fea6eb26 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:38.967032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:38.969043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:38.969445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:38.970021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] PATH_INFO: `/v3/projects/58b113ca5ca44542aa85e1a0df647724/users/d2081b72d6c349509e246c33bd136e79/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:38.998932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b113ca5ca44542aa85e1a0df647724, user_id=d2081b72d6c349509e246c33bd136e79, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:39.002075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:39.193147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1178/2365] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:38 2022] PUT /identity/v3/projects/58b113ca5ca44542aa85e1a0df647724/users/d2081b72d6c349509e246c33bd136e79/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:39.197914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:39.210182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607255e3-4294-450b-88d5-16f68dd30c84 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:39.278680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:39.279535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.279826 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.284783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.284783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:39.284783 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015694-1705-442d-90ba-59a61e93609c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:39.296699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1188/2366] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:39.298107 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:39.311172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84491a08-483e-434c-b2c0-3153008a431d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:39.319047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:39.320070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.320878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.321164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] PATH_INFO: `/v3/projects/58b113ca5ca44542aa85e1a0df647724/users/d2081b72d6c349509e246c33bd136e79/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.323286 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b113ca5ca44542aa85e1a0df647724, user_id=d2081b72d6c349509e246c33bd136e79, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:39.325994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:39.337487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1179/2367] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:39 2022] PUT /identity/v3/projects/58b113ca5ca44542aa85e1a0df647724/users/d2081b72d6c349509e246c33bd136e79/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:39.347115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:39.349061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.349362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.349646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.384141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] MFA Rules not processed for user `d2081b72d6c349509e246c33bd136e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:39.385619 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:39.462886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1189/2368] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:39.485547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:39.485547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:39.485968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:39.485968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:39.563201 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c3ea9-1112-40dd-ab79-1d1a03caa098 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:39.580800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:39.581193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127cb-f84b-4b78-bf3c-179e594899a0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:39.581481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.581801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.582088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.583978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:39.587004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:39.588032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.588497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.588842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] PATH_INFO: `/v3/projects/0c85e4ba7507467482af03e6e03a0b16` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.590163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c85e4ba7507467482af03e6e03a0b16)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:39.592375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:39.652326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:39.653874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd99d-3831-49e7-8338-6e34863f7e18 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:39.664324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1190/2369] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:39.788982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c85e4ba7507467482af03e6e03a0b16', 'request_id': 'req-0c151c08-d155-49c6-bf34-34f5cb766718'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:39.795369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c85e4ba7507467482af03e6e03a0b16', 'request_id': 'req-0c151c08-d155-49c6-bf34-34f5cb766718'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:39.801398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c85e4ba7507467482af03e6e03a0b16', 'request_id': 'req-0c151c08-d155-49c6-bf34-34f5cb766718'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:39.909979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] The token cache is being invalidate because project 0c85e4ba7507467482af03e6e03a0b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:39.910745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:39.911390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c151c08-d155-49c6-bf34-34f5cb766718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:39.912973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1180/2370] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:39 2022] DELETE /identity/v3/projects/0c85e4ba7507467482af03e6e03a0b16 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:39.920069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:39.920693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:39.920774 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:39.940837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:39.942143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.015542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1191/2371] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:40.022254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.022552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.022846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.042956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:40.044314 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.066761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1181/2372] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:40.067603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.077555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0febf4-922b-4484-92a3-f1ce6243057f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.086204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.086891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.087183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.087468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.088205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:40.088519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.090207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.104660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1192/2373] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:32:40.131311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.142291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e80e48-28a7-4730-a681-ddc26541acac None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.147918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.148809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.149230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.149613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.150896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.153224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.249948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.252062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.265821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bbb9c-1459-4bea-be9e-7ecf302bab43 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.270866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.271789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.272079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.272360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] PATH_INFO: `/v3/users/71c27adb4cf3437eaa795efa12deebe6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.274209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c27adb4cf3437eaa795efa12deebe6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.276200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.308763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1182/2374] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:40 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:40.313479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.375498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043df2-34ff-4e79-b17d-0089da626b4f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.380223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.380895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.381189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.381475 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.382208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.384172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.489908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1183/2375] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:40 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:40.496034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.500702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c27adb4cf3437eaa795efa12deebe6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:40.501728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c27adb4cf3437eaa795efa12deebe6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:40.508666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047a15-712d-42c9-a4cb-4a9aa91a1348 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.514388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.515112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.515394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.515707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.522469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.522469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.529125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1184/2376] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:40.533393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.535376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.550094 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabba7-bb7f-4286-8b2a-82eb4a5231ba admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.555099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.556220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.556220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.556389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.557185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.558880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.569873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1185/2377] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:40.574862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.575242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c27adb4cf3437eaa795efa12deebe6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:40.576031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c27adb4cf3437eaa795efa12deebe6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:40.585121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b379f052-7ea7-4eba-9d04-b64af37bc713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c27adb4cf3437eaa795efa12deebe6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:40.593508 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373fff2-01b2-46c0-9a67-53e18992ae79 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.598884 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.599857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.600213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.600567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] PATH_INFO: `/v3/projects/6315ff31c3974bfebab2db45c38695cd/users/c2d2fae2942e4d7da4d74ad2f48716d9/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.607861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1193/2378] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:40 2022] DELETE /identity/v3/users/71c27adb4cf3437eaa795efa12deebe6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:40.613570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.613570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.613570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.620856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6315ff31c3974bfebab2db45c38695cd, user_id=c2d2fae2942e4d7da4d74ad2f48716d9, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.623812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabae29-5e17-4331-89e5-91546b593b66 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.648313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:40.651136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.719212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1194/2379] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:40.729166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.730510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.778753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1186/2380] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:40 2022] PUT /identity/v3/projects/6315ff31c3974bfebab2db45c38695cd/users/c2d2fae2942e4d7da4d74ad2f48716d9/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:40.792252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.793000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.793289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.793938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc0732-622a-40c7-ad19-87f030af2525 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.826579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] MFA Rules not processed for user `c2d2fae2942e4d7da4d74ad2f48716d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:40.827670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.854920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.855836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.856133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.856416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:40.904742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1187/2381] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:40.917270 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:40.918669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:40.926150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:40.927648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:40.937888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1195/2382] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:40.984268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00925cdb-4662-4484-833d-2d4d4d4b31a1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:40.988659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:40.989332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:40.989673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:40.989993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:41.000621 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:41.002234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a1ae6-0034-469d-a9eb-e5ca31ad6e39 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:41.013126 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1188/2383] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:41.449283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:41.451094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:41.480996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba84a2-6960-4301-9d08-c519392333fb service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:41.487902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:41.498787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:41.499760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:41.500152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:41.500536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] PATH_INFO: `/v3/projects/f2329de0783643748124e381dfc01084` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:41.502426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2329de0783643748124e381dfc01084)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:41.504526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:41.641378 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2329de0783643748124e381dfc01084', 'request_id': 'req-722813a3-bc0e-4605-9559-b7fbb0397e22'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:41.647727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2329de0783643748124e381dfc01084', 'request_id': 'req-722813a3-bc0e-4605-9559-b7fbb0397e22'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:41.789080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2329de0783643748124e381dfc01084', 'request_id': 'req-722813a3-bc0e-4605-9559-b7fbb0397e22'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:41.807752 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] The token cache is being invalidate because project f2329de0783643748124e381dfc01084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:41.808303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:41.809172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-722813a3-bc0e-4605-9559-b7fbb0397e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:41.811397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1196/2384] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:41 2022] DELETE /identity/v3/projects/f2329de0783643748124e381dfc01084 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:41.816925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:41.818104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:41.826225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:41.852232 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:41.856876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:41.935565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1189/2385] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:41.943562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:41.944122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:41.944122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:41.989129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:41.990414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:42.004154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1197/2386] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:42.007474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.024101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cf99a-1a43-40b4-abc3-3d435bc1063f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.031376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.032326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.032645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.032990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.033922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:42.034377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.036362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.050837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1190/2387] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:42.055803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.072996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15928e49-8e8d-481e-b938-a2c0ab49fc36 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.088904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.089901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.090338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.090762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.091990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.094410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.214208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1198/2388] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:42 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:42.218384 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.228915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cfd6-3d82-4fe9-9a11-0e7201607774 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.288472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.289176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.289467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.289748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.294020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.294020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.436076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.437937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:42.448717 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1191/2389] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:42 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:42.452840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.454817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc94e1-ab48-4350-8aff-ad689694807b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.463024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.464051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.464203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd1a0c-4f84-4ead-b982-d141fa15dcf2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.464696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.465220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] PATH_INFO: `/v3/users/65b11b36fc81420a89aa5d650a089a40` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.466578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b11b36fc81420a89aa5d650a089a40)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.469089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.470143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.471034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.471601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.472213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.474393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:42.474794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.477698 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.494115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1192/2390] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:42.496020 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.514478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d6a86-fe9e-4299-8181-ff18559b423f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.520913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.521607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.521934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.522220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.523001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.525173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.536942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1193/2391] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:42.541159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.552331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79a5d6-4c43-4191-b39e-59a654d3ddf5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.559563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.560600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.560903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.561190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] PATH_INFO: `/v3/projects/c9c81f8de791403eb2d4b965e8535cb9/users/f8b8b206af104b068e6b11d5b37c5d14/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.578381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c81f8de791403eb2d4b965e8535cb9, user_id=f8b8b206af104b068e6b11d5b37c5d14, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.581065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c3152-87d3-420e-aec3-7322ae9febdb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.740496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1194/2392] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:42 2022] PUT /identity/v3/projects/c9c81f8de791403eb2d4b965e8535cb9/users/f8b8b206af104b068e6b11d5b37c5d14/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:42.752274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.752578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.752888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.784850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] MFA Rules not processed for user `f8b8b206af104b068e6b11d5b37c5d14`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:42.786361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:42.825591 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b11b36fc81420a89aa5d650a089a40'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:42.826495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b11b36fc81420a89aa5d650a089a40'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:42.884080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1195/2393] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:42.895562 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:42.897022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:42.957517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39620d03-1a51-4fd6-b86c-c56486940c67 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:42.962202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:42.962996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:42.963535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:42.964632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:42.986098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:42.986470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b11b36fc81420a89aa5d650a089a40'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:42.987057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b11b36fc81420a89aa5d650a089a40'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:42.988638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753aa45-6ffe-447d-a63e-a70c48e86633 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:42.997087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b11b36fc81420a89aa5d650a089a40'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.000819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1196/2394] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:43.020479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1199/2395] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:42 2022] DELETE /identity/v3/users/65b11b36fc81420a89aa5d650a089a40 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:43.025697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.026080 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.026364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.035031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:43.037078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.047122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:43.048559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.123106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624afd5-5cbf-4829-bc16-e34b353c2dfc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:43.128141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:43.129107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.129472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.129856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] PATH_INFO: `/v3/users/f8b8b206af104b068e6b11d5b37c5d14` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.131151 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b8b206af104b068e6b11d5b37c5d14)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:43.133762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:43.142040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1197/2396] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:43.153517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:43.159743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.224503 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc17d8-038e-4c20-bf46-d3a84a7faaff None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:43.228484 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:43.229152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.229429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.229910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.240187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:43.241635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a5f42-a077-43a7-ab39-b33eb47d9d9e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:43.253332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1198/2397] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:43.405537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b8b206af104b068e6b11d5b37c5d14'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.406220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b8b206af104b068e6b11d5b37c5d14'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.484059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:43.517778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b8b206af104b068e6b11d5b37c5d14'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.518447 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b8b206af104b068e6b11d5b37c5d14'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.526400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b8b206af104b068e6b11d5b37c5d14'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:43.546311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1200/2398] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:43 2022] DELETE /identity/v3/users/f8b8b206af104b068e6b11d5b37c5d14 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:43.551017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.551359 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.551646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.569638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:43.571744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.638911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1199/2399] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:43.648959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:43.650500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.711200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de1dbd-6d12-4af4-94bb-640f41d4b8c1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:43.715207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:43.715835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.716070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.716354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.727392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:43.734377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bdd11-4083-477b-b6ed-ede11519ef3a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:43.739713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1201/2400] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:43.853704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:43.856122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:43.870132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cfce5-3a40-4919-8316-581d98b5a1c3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:43.877362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:43.878481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:43.878913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:43.879530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] PATH_INFO: `/v3/projects/7bf98033e0e548d4b9977c0ee78a80cb` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:43.881099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf98033e0e548d4b9977c0ee78a80cb)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:43.883386 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:44.068512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf98033e0e548d4b9977c0ee78a80cb', 'request_id': 'req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.075628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf98033e0e548d4b9977c0ee78a80cb', 'request_id': 'req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.087568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf98033e0e548d4b9977c0ee78a80cb', 'request_id': 'req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.176546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] The token cache is being invalidate because project 7bf98033e0e548d4b9977c0ee78a80cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:44.182530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.182530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a87cbfc-c2c4-42bb-8bce-6e6442186fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.182530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1200/2401] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:43 2022] DELETE /identity/v3/projects/7bf98033e0e548d4b9977c0ee78a80cb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:44.185641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.186317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.186638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.214781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:44.216420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:44.279943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1202/2402] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:44.289104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899893-a465-4654-99e2-d7158e060506 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.289422 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899893-a465-4654-99e2-d7158e060506 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.289705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899893-a465-4654-99e2-d7158e060506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.308909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0a899893-a465-4654-99e2-d7158e060506 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:44.310192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0a899893-a465-4654-99e2-d7158e060506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:44.318916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1201/2403] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:44.323950 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:44.337893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaaa4b-4fd5-41bd-818b-4dd7c93a5855 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:44.344400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:44.345693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.346043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.346334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.347435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:44.347762 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:44.349463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:44.362712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1203/2404] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:32:44.367904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899893-a465-4654-99e2-d7158e060506 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:44.382164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899893-a465-4654-99e2-d7158e060506 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:44.388798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:44.389499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.389895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.390645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.391032 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:44.393305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:44.406767 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:44.408683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:44.430596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23422c06-bbf8-4762-a657-46ea1623adf8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:44.437516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:44.438651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.439012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.439312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] PATH_INFO: `/v3/projects/c9c81f8de791403eb2d4b965e8535cb9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.440909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c81f8de791403eb2d4b965e8535cb9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:44.443194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:44.661831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1202/2405] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:44 2022] POST /identity/v3/projects => generated 391 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:32:44.661831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:44.661831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:44.703389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c81f8de791403eb2d4b965e8535cb9', 'request_id': 'req-279ab1b1-b68e-4f7d-8a00-94f736538c67'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.712342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c81f8de791403eb2d4b965e8535cb9', 'request_id': 'req-279ab1b1-b68e-4f7d-8a00-94f736538c67'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.736594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ca00-d47b-41a2-9fed-1f3ec293da4e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:44.838741 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c81f8de791403eb2d4b965e8535cb9', 'request_id': 'req-279ab1b1-b68e-4f7d-8a00-94f736538c67'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.853261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] The token cache is being invalidate because project c9c81f8de791403eb2d4b965e8535cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:44.853734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.855037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:44.858536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1204/2406] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:44 2022] DELETE /identity/v3/projects/c9c81f8de791403eb2d4b965e8535cb9 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:44.860170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:44.861760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:44.950291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1b1-b68e-4f7d-8a00-94f736538c67 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:44.975687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:44.975827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:44.978500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.978994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:44.979298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.979587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] PATH_INFO: `/v3/users/c2d2fae2942e4d7da4d74ad2f48716d9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.979828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:44.980813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:44.984070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d2fae2942e4d7da4d74ad2f48716d9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:44.984288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:44.991108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:44.993497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.015791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:45.247739 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1205/2407] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:44 2022] POST /identity/v3/users => generated 296 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:45.250862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:45.261889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7df3f5-436d-40a5-9ca4-471c759af375 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:45.266654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:45.267322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.267612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.267906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.268635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:45.270269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.281258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1206/2408] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:45.285395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:45.295106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479db20-4b7d-4409-b085-7c8f2b39edf9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:45.299852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:45.300486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.300778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.301062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.301855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:45.303377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.314085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1207/2409] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:32:45.317933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:45.328221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d842a7-b77f-4d7f-b8c5-718bfc8be25b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:45.333204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:45.334111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.334403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.334688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] PATH_INFO: `/v3/projects/137e8463d7b245f79bc8c62d9510213d/users/2bbc6bf8c761408f9186b6881bd9a6a2/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.349839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=137e8463d7b245f79bc8c62d9510213d, user_id=2bbc6bf8c761408f9186b6881bd9a6a2, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:45.352494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fa00e-537e-451c-abcc-803d9626cbce admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.366574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d2fae2942e4d7da4d74ad2f48716d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:45.379137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d2fae2942e4d7da4d74ad2f48716d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:45.500550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d2fae2942e4d7da4d74ad2f48716d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:45.501185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d2fae2942e4d7da4d74ad2f48716d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:45.501923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae579415-2587-4c1f-b03a-f798e72d098b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d2fae2942e4d7da4d74ad2f48716d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:45.523324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1208/2410] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:45 2022] PUT /identity/v3/projects/137e8463d7b245f79bc8c62d9510213d/users/2bbc6bf8c761408f9186b6881bd9a6a2/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:32:45.527567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1203/2411] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:32:44 2022] DELETE /identity/v3/users/c2d2fae2942e4d7da4d74ad2f48716d9 => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:45.531251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.535790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.535790 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.536036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.536036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.536036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.555874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:45.557336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:45.582488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] MFA Rules not processed for user `2bbc6bf8c761408f9186b6881bd9a6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:45.584750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:45.656138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1204/2412] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:32:45.683569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:45.685789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:45.735917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1209/2413] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:45.746314 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:45.748382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:45.789379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4ef03-89d0-4a63-80b5-31b8467129f1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:45.800879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:45.801549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.801983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.802269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.813319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:45.819105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6af821-d59e-4e0d-8c42-9beb4e854e3b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:45.819280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.819905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:45.821994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:45.822567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:45.823074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:45.826737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1205/2414] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:45.832483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:45.833970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86d6bb-e8e2-4e6e-953d-111007c04144 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:45.844518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1210/2415] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:46.064223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:46.380227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:46.380227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:46.394023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a79fd-9148-4944-a9d1-26f8515d497d service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:46.399385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:46.400194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:46.400480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:46.400758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] PATH_INFO: `/v3/projects/6315ff31c3974bfebab2db45c38695cd` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:46.401989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6315ff31c3974bfebab2db45c38695cd)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:46.404829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:46.533423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6315ff31c3974bfebab2db45c38695cd', 'request_id': 'req-e240d305-b8e7-4750-af10-f1dec9f8ba0d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:46.540345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6315ff31c3974bfebab2db45c38695cd', 'request_id': 'req-e240d305-b8e7-4750-af10-f1dec9f8ba0d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:46.545890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6315ff31c3974bfebab2db45c38695cd', 'request_id': 'req-e240d305-b8e7-4750-af10-f1dec9f8ba0d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:46.688907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] The token cache is being invalidate because project 6315ff31c3974bfebab2db45c38695cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:32:46.689229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:46.689929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e240d305-b8e7-4750-af10-f1dec9f8ba0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:32:46.691604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1206/2416] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:32:46 2022] DELETE /identity/v3/projects/6315ff31c3974bfebab2db45c38695cd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:46.697042 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:46.697517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:46.697613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:46.716789 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:46.718604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:46.846915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1211/2417] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:46.855000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:46.855872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:46.856295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:46.892868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:46.896075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:46.906144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1207/2418] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:46.910442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:46.923706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf228-8d3f-4654-9024-2b10c1078976 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:46.929013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:46.929913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:46.930202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:46.930496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:46.931232 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:32:46.931545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:46.933666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:46.934611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:46.936223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:46.951161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1212/2419] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:32:46.958867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:46.988087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c1f75-5066-452d-bb11-941e34c68da5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:46.993895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:46.994632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:46.994917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:46.995197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:46.996051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:46.998354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.035555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fc278-d521-4089-9911-9eb17ef2bd41 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.039323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.039979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.040266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.040546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.042612 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:47.115888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.118524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.128550 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1208/2420] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:32:47.181971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1213/2421] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:32:46 2022] POST /identity/v3/projects => generated 365 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:47.182882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:47.193381 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add0b-c712-4871-bd1d-52a1fcfecd23 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.253074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:47.261034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.261751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.262179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.262467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.263224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.265159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.420006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1209/2422] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:32:47 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:32:47.424427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:47.435534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2ed3-bddf-472f-883e-d3a7080ab3e0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.440963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.441629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.441959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.442562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.442983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.444627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.456469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1214/2423] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:47.467818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:47.482687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc031e-d0da-48c2-8022-3c24f0555035 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.487517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.488180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.488470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.488874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.489646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.491293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbf5d2-3fc6-4daa-8136-1642e0e0dc43 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.501709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1210/2424] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:32:47.505959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:47.515854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f42e96-c3b0-4861-bc64-bf7008e9ab34 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.520658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.521557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.521944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.522237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] PATH_INFO: `/v3/projects/3fb02621cdaf46ccb5e955aa961f247a/users/cf08058b40ea4ae6be8b5f702cbefe62/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.538365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb02621cdaf46ccb5e955aa961f247a, user_id=cf08058b40ea4ae6be8b5f702cbefe62, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.541538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.645696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1215/2425] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:32:47 2022] PUT /identity/v3/projects/3fb02621cdaf46ccb5e955aa961f247a/users/cf08058b40ea4ae6be8b5f702cbefe62/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:32:47.656108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dcc70-01c6-4071-81fa-734561b7e247 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.656327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dcc70-01c6-4071-81fa-734561b7e247 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.656607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dcc70-01c6-4071-81fa-734561b7e247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.684915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d92dcc70-01c6-4071-81fa-734561b7e247 None None] MFA Rules not processed for user `cf08058b40ea4ae6be8b5f702cbefe62`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:47.686283 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d92dcc70-01c6-4071-81fa-734561b7e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:47.753889 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1211/2426] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:47.765051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:32:47.766824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:47.830339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837f2d1-ed57-4658-af08-5f51a52423ab admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:32:47.834075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:32:47.834765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:47.835053 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:47.835336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:47.844820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:32:47.846317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:32:47.856085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1216/2427] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:32:49.332932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:49.483820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:51.321305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:51.449297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:51.614327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98443a-3830-4a47-b7c9-7dd5eeeee730 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:32:51.614626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98443a-3830-4a47-b7c9-7dd5eeeee730 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:32:51.614907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98443a-3830-4a47-b7c9-7dd5eeeee730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:32:51.632449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-df98443a-3830-4a47-b7c9-7dd5eeeee730 None None] MFA Rules not processed for user `be6cba32a31a425e964d7222c3d6b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:32:51.634294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-df98443a-3830-4a47-b7c9-7dd5eeeee730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:32:51.689036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1212/2428] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:32:53.657094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:54.272790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:54.860599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:56.043916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:56.755981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:57.700949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:32:58.801384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:00.380424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:00.687130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:01.266337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:01.656158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:02.226660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:03.375037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:03.449740 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:03.451377 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:03.723705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0b666-414c-435f-b3ac-0a2dee39fd8b service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:03.729895 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:03.731603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:03.732074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:03.732371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] PATH_INFO: `/v3/users/b22dad1e58e64db5a33b8eeaa97e1032` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:03.733743 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22dad1e58e64db5a33b8eeaa97e1032)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:03.735749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:03.947882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22dad1e58e64db5a33b8eeaa97e1032'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:03.950908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22dad1e58e64db5a33b8eeaa97e1032'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:04.119761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22dad1e58e64db5a33b8eeaa97e1032'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:04.120680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22dad1e58e64db5a33b8eeaa97e1032'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:04.149271 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22dad1e58e64db5a33b8eeaa97e1032'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:04.188811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1217/2429] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:03 2022] DELETE /identity/v3/users/b22dad1e58e64db5a33b8eeaa97e1032 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:04.193636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:04.202556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:04.202556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:04.248023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:04.249524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:04.365006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1213/2430] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:04.370734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:04.378472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:04.380511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:04.470917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ced4-94e1-4044-b98b-9765543cf87b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:04.474714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:04.475443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:04.475860 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:04.476176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:04.486712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:04.488269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:04.500239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1218/2431] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:04.940885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:04.942378 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:04.955630 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34439a66-ae9f-4d5d-b214-a9e7a57fcd5f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:04.979548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:04.981008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:04.981383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:04.981926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] PATH_INFO: `/v3/projects/de39d044f9cc47ddb76740368b3f1e74` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:04.983308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de39d044f9cc47ddb76740368b3f1e74)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:04.985427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:05.151576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de39d044f9cc47ddb76740368b3f1e74', 'request_id': 'req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.158202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de39d044f9cc47ddb76740368b3f1e74', 'request_id': 'req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.163875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de39d044f9cc47ddb76740368b3f1e74', 'request_id': 'req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.275603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] The token cache is being invalidate because project de39d044f9cc47ddb76740368b3f1e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:05.276173 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.277719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.288527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1214/2432] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:04 2022] DELETE /identity/v3/projects/de39d044f9cc47ddb76740368b3f1e74 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:05.293260 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:05.315204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd375d-3991-4030-86a2-b29fe5e6d3bf service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:05.379916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:05.380817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:05.381101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:05.381381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] PATH_INFO: `/v3/users/d2081b72d6c349509e246c33bd136e79` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:05.382684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2081b72d6c349509e246c33bd136e79)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:05.385222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:05.599885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2081b72d6c349509e246c33bd136e79'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.602370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2081b72d6c349509e246c33bd136e79'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.721357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2081b72d6c349509e246c33bd136e79'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.722012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2081b72d6c349509e246c33bd136e79'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.730173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbc9bca3-b842-454c-8753-cd8765bd5c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2081b72d6c349509e246c33bd136e79'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:05.747379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1219/2433] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:05 2022] DELETE /identity/v3/users/d2081b72d6c349509e246c33bd136e79 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:33:06.193574 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:06.195204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:06.209141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f80c1-7e41-418a-b456-f08fb7c9b16b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:06.266970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:06.267885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.268196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.268462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] PATH_INFO: `/v3/projects/58b113ca5ca44542aa85e1a0df647724` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.269645 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b113ca5ca44542aa85e1a0df647724)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:06.271593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:06.403549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:06.408325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b113ca5ca44542aa85e1a0df647724', 'request_id': 'req-0dc38e13-54dd-4cca-950b-2f460ae91ca3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:06.422866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b113ca5ca44542aa85e1a0df647724', 'request_id': 'req-0dc38e13-54dd-4cca-950b-2f460ae91ca3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:06.422866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b113ca5ca44542aa85e1a0df647724', 'request_id': 'req-0dc38e13-54dd-4cca-950b-2f460ae91ca3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:06.506932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] The token cache is being invalidate because project 58b113ca5ca44542aa85e1a0df647724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:06.506932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:06.506932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0dc38e13-54dd-4cca-950b-2f460ae91ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:06.508461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1215/2434] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:06 2022] DELETE /identity/v3/projects/58b113ca5ca44542aa85e1a0df647724 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:06.518794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.518794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.518794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.539080 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:06.540406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:06.601868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1220/2435] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:06.612166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.612456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.612740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.633071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:06.634334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:06.644445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1216/2436] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:06.649647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:06.671593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6942b-6dc3-474b-8cff-0ebf764bc03e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:06.675402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:06.677728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.678430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.679360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.682025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:33:06.682871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:06.686152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:06.708073 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1221/2437] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:33:06.713019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:06.722954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460148-9a70-410c-86f6-0983a25c794d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:06.728802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:06.729547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.731122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.731122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.731643 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:06.733466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:06.810506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1217/2438] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:33:06 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:33:06.814742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:06.815920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:06.828263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de2b3a-22f9-4680-bfa8-5d64b9819bed admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:06.891917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:06.892635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:06.892926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:06.893207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:06.893939 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:06.896022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:06.996429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1222/2439] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:33:06 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:07.000748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:07.014026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cf40-a2d4-4007-8bd9-b5ffb77b276b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:07.018633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:07.019301 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:07.019588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:07.019871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:07.020613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:07.022377 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:07.033917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1218/2440] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:07.037819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:07.052009 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561b320-84fa-4f45-bdc8-25338447e20f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:07.057292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:07.057999 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:07.058290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:07.068227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:07.068227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:07.068227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dcb6f-2a8f-430b-a2d2-1f08ea3ea1fe admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:07.074904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1223/2441] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:07.079308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:07.091279 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e53e2-5616-402d-9239-1c6dabc3625d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:07.096035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:07.096918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:07.097203 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:07.097480 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] PATH_INFO: `/v3/projects/f58a388e3d8143edb4132f42065d02f4/users/d9aedde1d62b40628d883d106beabcf3/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:07.114186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58a388e3d8143edb4132f42065d02f4, user_id=d9aedde1d62b40628d883d106beabcf3, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:07.117153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:07.189837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1219/2442] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:33:07 2022] PUT /identity/v3/projects/f58a388e3d8143edb4132f42065d02f4/users/d9aedde1d62b40628d883d106beabcf3/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:07.203211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:07.203685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:07.203978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:07.235150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] MFA Rules not processed for user `d9aedde1d62b40628d883d106beabcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:07.236464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:07.309411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1224/2443] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:07.321160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:07.322903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:07.387336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cfaed-8dcd-4c4b-8ced-4124453c8149 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:07.391294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:07.392013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:07.392305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:07.392594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:07.402156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:07.403633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7bf5d-72b6-48c2-8c7e-5cc3c53e70cb service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:07.412757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1220/2444] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:08.100900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:08.135536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:08.900877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:09.234874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:09.467171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:10.306623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:10.373138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:11.352948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:11.830925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:12.295059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:12.568939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:12.761358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:13.133231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:13.323292 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:13.438125 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:13.630058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:13.843280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:14.115661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:14.149800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:14.413032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:14.511048 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:15.117851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:15.310116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:15.443852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:15.518137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:15.519611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:15.584626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be063e5c-dcef-40ac-9e21-950d9fb5e731 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:15.589578 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:15.590479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:15.590772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:15.591056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] PATH_INFO: `/v3/users/2bbc6bf8c761408f9186b6881bd9a6a2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:15.592277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bbc6bf8c761408f9186b6881bd9a6a2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:15.594221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:15.880329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bbc6bf8c761408f9186b6881bd9a6a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:15.880994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bbc6bf8c761408f9186b6881bd9a6a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:15.921532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:15.991781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bbc6bf8c761408f9186b6881bd9a6a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:15.992497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bbc6bf8c761408f9186b6881bd9a6a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:16.001047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bbc6bf8c761408f9186b6881bd9a6a2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:16.020466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1225/2445] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:15 2022] DELETE /identity/v3/users/2bbc6bf8c761408f9186b6881bd9a6a2 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:16.025669 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:16.025993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:16.026305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:16.043458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:16.044796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:16.104072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1221/2446] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:16.114280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:16.115693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:16.176597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa25e-ec30-4db2-ad30-2d730a469622 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:16.184522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:16.184522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:16.184522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:16.184522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:16.191438 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:16.192902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:16.204109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1226/2447] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:33:16.481854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:16.855217 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:16.857329 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:16.876213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31960-4252-46fa-9471-880e6d2200e5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:16.881904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:16.884826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:16.884826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:16.884826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] PATH_INFO: `/v3/projects/137e8463d7b245f79bc8c62d9510213d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:16.886419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=137e8463d7b245f79bc8c62d9510213d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:16.890808 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:17.036909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137e8463d7b245f79bc8c62d9510213d', 'request_id': 'req-d3d95203-4e4e-470c-a199-f690eb4b6c8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.044587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137e8463d7b245f79bc8c62d9510213d', 'request_id': 'req-d3d95203-4e4e-470c-a199-f690eb4b6c8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.050742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137e8463d7b245f79bc8c62d9510213d', 'request_id': 'req-d3d95203-4e4e-470c-a199-f690eb4b6c8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.123940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:17.185431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] The token cache is being invalidate because project 137e8463d7b245f79bc8c62d9510213d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:17.185653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.186304 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d3d95203-4e4e-470c-a199-f690eb4b6c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.187957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1222/2448] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:16 2022] DELETE /identity/v3/projects/137e8463d7b245f79bc8c62d9510213d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:17.205034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:17.266252 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:17.268144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:17.296957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e380f-b8a0-4095-8864-bd211a8ce37a service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:17.378149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:17.379152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:17.379524 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:17.379870 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] PATH_INFO: `/v3/users/d9aedde1d62b40628d883d106beabcf3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:17.381785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9aedde1d62b40628d883d106beabcf3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:17.384544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:17.625930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9aedde1d62b40628d883d106beabcf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.626666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9aedde1d62b40628d883d106beabcf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.740045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9aedde1d62b40628d883d106beabcf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.740671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9aedde1d62b40628d883d106beabcf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.748294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9aedde1d62b40628d883d106beabcf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:17.767210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1227/2449] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:17 2022] DELETE /identity/v3/users/d9aedde1d62b40628d883d106beabcf3 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:17.771858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:17.772153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:17.772431 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:17.791793 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:17.793051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:17.862846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1223/2450] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:17.869701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:17.871202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:17.944905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19925-5edc-4c87-9fae-81afb114d862 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:17.952652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:17.953312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:17.953674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:17.954016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:17.964795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:17.966301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790bbb2-6410-43bb-b64c-7c353d7dc7fa service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:17.976893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1228/2451] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:18.016034 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:18.636530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:18.638002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:18.650269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9f62-526e-412b-98a6-32bd2aa84839 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:18.655077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:18.655936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:18.656224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:18.656708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] PATH_INFO: `/v3/projects/f58a388e3d8143edb4132f42065d02f4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:18.657851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58a388e3d8143edb4132f42065d02f4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:18.659691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:18.715813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:18.801801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58a388e3d8143edb4132f42065d02f4', 'request_id': 'req-c528e070-2fb4-48eb-ad0a-191caf73e4f6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:18.808576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58a388e3d8143edb4132f42065d02f4', 'request_id': 'req-c528e070-2fb4-48eb-ad0a-191caf73e4f6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:18.814250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58a388e3d8143edb4132f42065d02f4', 'request_id': 'req-c528e070-2fb4-48eb-ad0a-191caf73e4f6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:18.892280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] The token cache is being invalidate because project f58a388e3d8143edb4132f42065d02f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:18.892479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:18.893112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c528e070-2fb4-48eb-ad0a-191caf73e4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:18.894760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1224/2452] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:18 2022] DELETE /identity/v3/projects/f58a388e3d8143edb4132f42065d02f4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:18.900263 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:18.900570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:18.900845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:18.921620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:18.922880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:18.957906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:18.990434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1229/2453] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:18.997253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:18.997754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:18.998247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.026015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:19.026015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:19.038155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1225/2454] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:19.039920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.052005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72f071-501b-4b8d-9f50-297f45825740 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.071719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.072677 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.073056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.074897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.077928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:33:19.078363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.080521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.100541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1230/2455] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:33:19.107802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.120729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649575f-39e0-4735-8b65-ff80f047c7fe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.125926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.127146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.127737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.128167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.129962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.132245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.225958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1226/2456] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:33:19 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:19.226519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.240897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9ac61-4a46-45e9-86ff-d36ac7aba0dc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.344970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.346002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.346679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.346806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.356328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.364108 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.487694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1231/2457] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:33:19 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:33:19.491802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.502494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf6fc-3160-4e86-9860-2b152ea63e12 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.506957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.507609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.507891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.508158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.508896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.511002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.524260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1227/2458] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:19.526511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.536986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb13d6-c304-4ff6-ae1b-d544fea2323b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.544255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.545017 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.545226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.545510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.546443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.548087 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626cc1b-ccc9-4d97-8739-d25dfb545eff admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.558899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1232/2459] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:19.564515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.574529 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6fe19-48e8-463f-b28e-ab14eae736d7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.579309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.580182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.580467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.580744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] PATH_INFO: `/v3/projects/71e57e0bf8064c41b27e96f8f9b861ab/users/c17348af47604b57b4a0384b816b4151/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.595332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e57e0bf8064c41b27e96f8f9b861ab, user_id=c17348af47604b57b4a0384b816b4151, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.598081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.699768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1228/2460] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:33:19 2022] PUT /identity/v3/projects/71e57e0bf8064c41b27e96f8f9b861ab/users/c17348af47604b57b4a0384b816b4151/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:19.706005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.706356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.706696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.735731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] MFA Rules not processed for user `c17348af47604b57b4a0384b816b4151`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:19.736954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:19.801624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1233/2461] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:19.813264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:19.815334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:19.878300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f4ce-d992-4070-9a15-7dc5a79d211e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:19.882265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:19.882972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:19.888928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:19.888928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:19.892451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:19.893989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:19.904248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1229/2462] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:20.123272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:20.232253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:20.832513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:20.834028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:20.901695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d7e5e-f6ad-41da-80cc-ca726170a5fe None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:20.905593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:20.906286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:20.906580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:20.906866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:20.908857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:20.985857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:20.987306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aa706-8482-444e-ae99-e0bf64a030bf service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:20.998766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1234/2463] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:33:21.039887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:21.626269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:21.694618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:21.813092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:21.841548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.146564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.249168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.547123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.601742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.832365 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.878265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:22.932851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:23.397936 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:23.666685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:23.723444 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:24.100648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:24.184119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:24.750479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:24.819268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.128828 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.271180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.340796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.510469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.567184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:25.815600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:26.140732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:26.352632 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:26.847460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:26.906226 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.203917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.277902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.551905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.586102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.700486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.872256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:27.952215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.225908 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.300404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.454676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.597485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.660325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.947072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:28.989993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:29.251957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:29.470334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:29.602772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:29.604288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:29.616227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:29.672457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc7f3-b806-43f2-9329-2d1e734b4f53 service glance] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:29.678656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:29.682591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:29.682591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:29.682591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] PATH_INFO: `/v3/users/edf5a1f6e3584004b72af3b40c0b4e2d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:29.682591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf5a1f6e3584004b72af3b40c0b4e2d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:29.684564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:29.706382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:29.964604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf5a1f6e3584004b72af3b40c0b4e2d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:29.977210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf5a1f6e3584004b72af3b40c0b4e2d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:29.980323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.065298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf5a1f6e3584004b72af3b40c0b4e2d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.065527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf5a1f6e3584004b72af3b40c0b4e2d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.066167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf5a1f6e3584004b72af3b40c0b4e2d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.080041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1230/2464] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:29 2022] DELETE /identity/v3/users/edf5a1f6e3584004b72af3b40c0b4e2d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:30.085786 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:30.086254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:30.086654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:30.105104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:30.106768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:30.158967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.168468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1235/2465] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:30.179887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:30.182119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:30.190598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.241058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb227b-836f-4bc8-b086-7aee51141da4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:30.245541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:30.250327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:30.250327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:30.250327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:30.255604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:30.257044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf491c4-8e2d-4dd3-ab78-073732763676 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:30.267168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1231/2466] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:33:30.295026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.327501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.723622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.730288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:30.733423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:30.746600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cc743-003e-4326-871d-d74e783217f7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:30.751449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:30.752329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:30.752620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:30.752930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] PATH_INFO: `/v3/projects/5257e4d00029401298010cf0637bbcc8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:30.754149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5257e4d00029401298010cf0637bbcc8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:30.761255 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:30.786044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.861573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5257e4d00029401298010cf0637bbcc8', 'request_id': 'req-cbd11b34-171c-4188-bdb0-d21c72435128'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.868035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5257e4d00029401298010cf0637bbcc8', 'request_id': 'req-cbd11b34-171c-4188-bdb0-d21c72435128'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.946712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5257e4d00029401298010cf0637bbcc8', 'request_id': 'req-cbd11b34-171c-4188-bdb0-d21c72435128'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.958020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] The token cache is being invalidate because project 5257e4d00029401298010cf0637bbcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:30.962667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.962667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbd11b34-171c-4188-bdb0-d21c72435128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:30.962667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1236/2467] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:30 2022] DELETE /identity/v3/projects/5257e4d00029401298010cf0637bbcc8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:30.963030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:30.966370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:30.966666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:30.966949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:30.985789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:30.987067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:31.056306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1232/2468] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:31.064985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.065279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.065562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.070223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.084568 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:31.085896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:31.095312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1237/2469] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:33:31.099590 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.109597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c6364-e251-416a-aad6-c032aa65cdae None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.114186 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.114831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.115127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.115437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.116168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:33:31.116479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.118627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.130181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1233/2470] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:33:31.135120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.144526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d48fbe-4768-477b-90ce-e80478eb75d5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.149176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.149991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.150253 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.150528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.151230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.153120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.196563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.285196 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1238/2471] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:33:31 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:31.290025 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.300059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40854ca-47ff-4134-8e41-d29c825baf18 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.363190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.363190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.363190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.363190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.363601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.365117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.392083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.466248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1234/2472] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:33:31 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:31.470498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.481459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8097-d262-4f6a-b037-e96894509dee admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.486275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.486980 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.487272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.487557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.488296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.489966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.505239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1239/2473] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:31.506873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.517175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e75fa-3832-4e92-b648-4149cceaf04a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.521942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.522597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.522916 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.523197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.523969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.525566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469cabc-656e-4e0d-9e37-b4d4fefd7099 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.536436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1235/2474] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:31.541114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.551342 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dfdef-58b4-484b-b59e-bdc398841990 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.554135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82,keystone:INSERT=2,keystone:DELETE=16 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.558933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.559840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.563778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.563778 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] PATH_INFO: `/v3/projects/ceeb3ddff16645c6bd9ca8e138ed356f/users/a1336e374bb542b7a036e3c0d0af54f3/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.575878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceeb3ddff16645c6bd9ca8e138ed356f, user_id=a1336e374bb542b7a036e3c0d0af54f3, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.579155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.580317 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.637945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1240/2475] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:33:31 2022] PUT /identity/v3/projects/ceeb3ddff16645c6bd9ca8e138ed356f/users/a1336e374bb542b7a036e3c0d0af54f3/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:31.643408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229957-42e8-4943-a249-67710eb20f4f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.644168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229957-42e8-4943-a249-67710eb20f4f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.644168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229957-42e8-4943-a249-67710eb20f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.662828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b2229957-42e8-4943-a249-67710eb20f4f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:31.664551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b2229957-42e8-4943-a249-67710eb20f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:31.722685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1236/2476] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:31.733326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:31.735352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:31.795606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.799698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab44c6-4eb5-471c-8563-63fcb86412df admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:31.803449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:31.804124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:31.804406 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:31.804681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:31.813916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:31.815327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:31.820474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:31.825061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1241/2477] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:32.100574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:32.140410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:32.491546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:32.601188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:33.168991 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:33.259122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749615-eec7-4220-9368-6868e13008fd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:33.263466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749615-eec7-4220-9368-6868e13008fd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:33.263466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749615-eec7-4220-9368-6868e13008fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:33.295154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e1749615-eec7-4220-9368-6868e13008fd None None] MFA Rules not processed for user `a1336e374bb542b7a036e3c0d0af54f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:33.296418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e1749615-eec7-4220-9368-6868e13008fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:33.370348 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1237/2478] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:33:33.381194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:33.382679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:33.442839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0bfcf-5dd2-488e-866c-a37d084fe123 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:33.446700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:33.447363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:33.447651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:33.448056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:33.457229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:33.458718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:33.468812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1242/2479] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:33.876345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:34.438186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:34.992423 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:35.254992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:35.438005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:35.438552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:35.439010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:35.439122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:35.445497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:35.458247 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] MFA Rules not processed for user `a1336e374bb542b7a036e3c0d0af54f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:35.459560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:35.466624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1238/2480] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:35.658277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:35.779296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:45.629399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:45.957655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=39 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:47.792427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:47.793994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:47.863688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160721a-0a39-489e-a606-21a6af464d68 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:47.868608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:47.869258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:47.869545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:47.869853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:47.871721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:47.884367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:47.885966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263e52a-6784-4322-8b5f-a30f1eee33c8 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:47.896195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1243/2481] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:52.089058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:52.090581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:52.104137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095462-6f5d-4899-9957-d8639a1d615b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:52.109106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:52.110014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:52.110325 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:52.110613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] PATH_INFO: `/v3/users/cf08058b40ea4ae6be8b5f702cbefe62` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:52.111877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf08058b40ea4ae6be8b5f702cbefe62)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:52.113886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:52.382785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf08058b40ea4ae6be8b5f702cbefe62'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:52.394895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf08058b40ea4ae6be8b5f702cbefe62'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:52.465583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf08058b40ea4ae6be8b5f702cbefe62'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:52.466236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf08058b40ea4ae6be8b5f702cbefe62'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:52.466877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf08058b40ea4ae6be8b5f702cbefe62'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:52.479534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1239/2482] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:33:52 2022] DELETE /identity/v3/users/cf08058b40ea4ae6be8b5f702cbefe62 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:52.484428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:52.487585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:52.487585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:52.502812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:52.504055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:52.562347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1244/2483] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:52.572710 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:52.574163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:52.629560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714edef5-e556-40a6-9561-9d346451de19 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:52.633856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:52.634439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:52.634726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:52.635004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:52.644287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:52.645751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b4eac-77f9-4ee8-b721-72bc7f58b2ca service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:52.655131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1240/2484] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:53.061995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.063373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:53.075601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd855-4a07-434d-a731-92c01effb3a1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.095674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.095674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.095674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.095674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] PATH_INFO: `/v3/projects/3fb02621cdaf46ccb5e955aa961f247a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.096107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb02621cdaf46ccb5e955aa961f247a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.096107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.203177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb02621cdaf46ccb5e955aa961f247a', 'request_id': 'req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:53.213738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb02621cdaf46ccb5e955aa961f247a', 'request_id': 'req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:53.277706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb02621cdaf46ccb5e955aa961f247a', 'request_id': 'req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:53.315116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] The token cache is being invalidate because project 3fb02621cdaf46ccb5e955aa961f247a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:33:53.316024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:53.316168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a414650f-7cd4-4196-b6a9-2f90ffc90ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:33:53.317909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1245/2485] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:33:53 2022] DELETE /identity/v3/projects/3fb02621cdaf46ccb5e955aa961f247a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:53.323178 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.324347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.325296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.351055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:53.352263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:53.414949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1241/2486] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:53.423323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.423617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.423929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.443022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:53.444748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:53.452982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1246/2487] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:33:53.461083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.471036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea4a9b-2345-4252-affe-78d428b28c7e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.475647 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.476328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.476615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.476892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.477624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:33:53.477967 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.479555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.492567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1242/2488] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:33:53.497034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.507225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53f4f-138b-42db-9e5a-b320051eff04 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.511959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.515008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.515008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.515008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.515008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.515829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.519124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:53.610490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1247/2489] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:33:53 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:53.614656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.619984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:33:53.625860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec715c29-4de2-4445-8537-5a1eec373b1b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.690326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.690877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.691224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.691451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.692179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.694144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.875598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1243/2490] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:33:53 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:33:53.880668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.892165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552280-3134-40ba-bfc8-dd8e24f5200c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.897729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.898615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.898913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.899223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.899947 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.901550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.912382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1248/2491] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:53.916356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.926448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643dad-e024-4bfc-9c4b-e29b48764b65 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.931724 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.932442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.932731 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.933013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.933819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.935512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede8e54-dd54-4390-90e8-9d78ace6524d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:53.946300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1244/2492] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:33:53.950580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:53.960469 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cdea1-1326-4173-be67-db8a9babce83 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:53.965728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:53.966343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:53.966627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:53.966911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] PATH_INFO: `/v3/projects/8ae0529df8aa4640806b4e7f3680e87b/users/73045af76072450a999de6d3fc5962e6/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:53.982575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae0529df8aa4640806b4e7f3680e87b, user_id=73045af76072450a999de6d3fc5962e6, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:53.985245 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:54.083531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1249/2493] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:33:53 2022] PUT /identity/v3/projects/8ae0529df8aa4640806b4e7f3680e87b/users/73045af76072450a999de6d3fc5962e6/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:33:54.088591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9576-f701-4792-8a95-4ee5d5cad469 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:54.088877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9576-f701-4792-8a95-4ee5d5cad469 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:54.089151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9576-f701-4792-8a95-4ee5d5cad469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:54.107047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-472b9576-f701-4792-8a95-4ee5d5cad469 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:54.109132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-472b9576-f701-4792-8a95-4ee5d5cad469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:54.169413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1245/2494] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:54.178525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:54.180805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:54.237504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15271-dc0c-4716-a628-104ff1e18087 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:54.241237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:54.242424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:54.242718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:54.243004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:54.252129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:54.253562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:33:54.263138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1250/2495] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:33:59.717704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:59.718233 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:59.718548 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:59.780269 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] MFA Rules not processed for user `73045af76072450a999de6d3fc5962e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:33:59.782531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:33:59.883223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1246/2496] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:33:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:33:59.883888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:33:59.895487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6737072-eddc-4470-9af9-d8f9d8a0b87c service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:33:59.900919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:33:59.901590 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:33:59.902098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:33:59.902380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:33:59.903287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:33:59.905213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.039225 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1251/2497] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:33:59 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:34:00.043489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:00.056635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f006b-62c7-45d2-b0e0-56356e5c791a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:00.065566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:00.132534 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:00.133352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:00.133715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:00.134097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:00.134871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:00.137047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.251248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1247/2498] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:34:00 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:34:00.255458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:00.266470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f6c8-9bdb-463d-8842-f998b9306e7e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:00.271048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:00.271714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:00.272002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:00.272324 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:00.273163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:00.277078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.289019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1252/2499] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:34:00.293375 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:00.304539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d964fb-e3aa-4f39-88ba-fc1913fd8cce admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:00.309739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:00.310701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:00.310990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:00.311277 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] PATH_INFO: `/v3/projects/5618969e1eb64ef3af8d03b451da171f/users/88604a38aa3e438ea521ee50393c94b2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:00.328300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5618969e1eb64ef3af8d03b451da171f, user_id=88604a38aa3e438ea521ee50393c94b2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:00.331383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.408841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1248/2500] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:34:00 2022] PUT /identity/v3/projects/5618969e1eb64ef3af8d03b451da171f/users/88604a38aa3e438ea521ee50393c94b2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:00.413374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:00.429737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77a72-bc0d-4878-bae5-abcf73d530c4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:00.490799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:00.491541 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:00.491831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:00.492113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:00.492854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:00.494652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.506337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1253/2501] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:34:00.510243 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:00.522326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46321-9466-430a-b971-eccc892be4e2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:00.527594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:00.527911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:00.528190 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:00.528467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] PATH_INFO: `/v3/projects/5618969e1eb64ef3af8d03b451da171f/users/88604a38aa3e438ea521ee50393c94b2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:00.531302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5618969e1eb64ef3af8d03b451da171f, user_id=88604a38aa3e438ea521ee50393c94b2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:00.534319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc530f-c61b-4ec1-b9d1-c4c87076443e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:00.547770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1249/2502] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:34:00 2022] PUT /identity/v3/projects/5618969e1eb64ef3af8d03b451da171f/users/88604a38aa3e438ea521ee50393c94b2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:34:01.173616 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:01.175466 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:01.188243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b16064-4e2e-4a08-af9e-2456178dbf7b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:01.202810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:01.252551 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:01.253432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:01.253717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:01.254027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] PATH_INFO: `/v3/users/c17348af47604b57b4a0384b816b4151` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:01.255228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c17348af47604b57b4a0384b816b4151)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:01.257669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:01.464429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c17348af47604b57b4a0384b816b4151'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:01.465527 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c17348af47604b57b4a0384b816b4151'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:01.598111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c17348af47604b57b4a0384b816b4151'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:01.599468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c17348af47604b57b4a0384b816b4151'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:01.609266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c17348af47604b57b4a0384b816b4151'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:01.640570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1254/2503] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:34:01 2022] DELETE /identity/v3/users/c17348af47604b57b4a0384b816b4151 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:01.645379 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:01.645912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:01.646416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:01.666718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:01.667898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:01.742617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1250/2504] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:01.752629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:01.756718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:01.819089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85122-9f23-4bd0-965e-5496026caeba admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:01.825373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:01.826065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:01.830905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:01.830905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:01.851105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:01.853995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09b7c-afea-4307-8310-5980b20eddb1 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:01.867888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1255/2505] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:34:02.774924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:02.776412 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:02.798388 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8aeec-6922-454c-8736-93635615a6d3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:02.809231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:02.811467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:02.813723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:02.813723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] PATH_INFO: `/v3/projects/71e57e0bf8064c41b27e96f8f9b861ab` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:02.814564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e57e0bf8064c41b27e96f8f9b861ab)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:02.817783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:02.955970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e57e0bf8064c41b27e96f8f9b861ab', 'request_id': 'req-542ac03c-263e-44db-ae00-ab9e0a547b7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:02.968770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e57e0bf8064c41b27e96f8f9b861ab', 'request_id': 'req-542ac03c-263e-44db-ae00-ab9e0a547b7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:02.985396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:02.994852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e57e0bf8064c41b27e96f8f9b861ab', 'request_id': 'req-542ac03c-263e-44db-ae00-ab9e0a547b7b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:03.164960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] The token cache is being invalidate because project 71e57e0bf8064c41b27e96f8f9b861ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:34:03.165504 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:03.167426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-542ac03c-263e-44db-ae00-ab9e0a547b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:03.171549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1251/2506] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:34:02 2022] DELETE /identity/v3/projects/71e57e0bf8064c41b27e96f8f9b861ab => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:03.176206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:03.176733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:03.176848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:03.225700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:03.226971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:03.306685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:03.402781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1256/2507] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:03.402890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:03.402890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:03.403163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:03.439956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:03.439956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:03.466981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1252/2508] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:03.468204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:03.492126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07019285-dbc3-447f-9a95-9677186ca194 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:03.504078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:03.504666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:03.509880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:03.509880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:03.510420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:34:03.510684 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:03.512686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:03.548658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1257/2509] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:34:03.554450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:03.569980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979a45-4613-49e7-b3fa-aabe083d697a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:03.575106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:03.575783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:03.576073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:03.576418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:03.577159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:03.579341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:03.701108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1253/2510] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:34:03 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:34:03.705587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:03.726066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbcaef-a141-450b-b662-0e38105452d5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:03.858788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:03.868310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:03.868310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:03.868310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:03.868310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:03.868310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:04.019325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1258/2511] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:34:03 2022] POST /identity/v3/users => generated 288 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:34:04.020774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:04.057539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da983449-25af-4ecf-92df-1a426254033b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:04.085557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:04.088407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:04.088930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:04.091552 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:04.092615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:04.098218 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:04.117613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1254/2512] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:34:04.121613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:04.153975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabe4ae-6f94-4f8e-9620-84639c321eca admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:04.170633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:04.176089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:04.176434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:04.176965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:04.177633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:04.179354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda2fb4-9ca1-4f8d-b5f7-1b85e74af0b6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:04.208638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1259/2513] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:34:04.214298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:04.240278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b3b72-4ee7-43d4-b23b-2055ecc71c47 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:04.245999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:04.248071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:04.252293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:04.253016 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] PATH_INFO: `/v3/projects/50add55111c4427d94f06a3cf0555067/users/4108ab2452684c819e93dabd4f9e7c98/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:04.284215 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50add55111c4427d94f06a3cf0555067, user_id=4108ab2452684c819e93dabd4f9e7c98, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:04.287537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:04.360578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1255/2514] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:34:04 2022] PUT /identity/v3/projects/50add55111c4427d94f06a3cf0555067/users/4108ab2452684c819e93dabd4f9e7c98/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:04.372063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0effb9-f9dd-49a9-b1dc-e09a1dad7f8a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:04.372366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0effb9-f9dd-49a9-b1dc-e09a1dad7f8a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:04.372647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0effb9-f9dd-49a9-b1dc-e09a1dad7f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:04.414706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1b0effb9-f9dd-49a9-b1dc-e09a1dad7f8a None None] MFA Rules not processed for user `4108ab2452684c819e93dabd4f9e7c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:04.416102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1b0effb9-f9dd-49a9-b1dc-e09a1dad7f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:04.527512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1260/2515] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:04.538718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:04.540893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:04.632952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138526-9c88-47a0-a0de-7ea433bc4810 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:04.640870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:04.641545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:04.641880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:04.642161 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:04.654633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:04.656123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917e8d4-1826-4b9f-bbd8-cf2f2080dca7 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:04.668547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1256/2516] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:34:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:34:06.660299 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:06.660800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:06.660898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:06.680558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] MFA Rules not processed for user `4108ab2452684c819e93dabd4f9e7c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:06.681901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:06.689259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1261/2517] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:07.132330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363241e5-3a78-4696-856c-efc572e6f0fb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:07.133255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363241e5-3a78-4696-856c-efc572e6f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:07.133729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363241e5-3a78-4696-856c-efc572e6f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:07.170678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-363241e5-3a78-4696-856c-efc572e6f0fb None None] MFA Rules not processed for user `88604a38aa3e438ea521ee50393c94b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:07.172573 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-363241e5-3a78-4696-856c-efc572e6f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:07.278413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1257/2518] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:07.284442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:07.285948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:07.334312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:07.372672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55f9b-5e6a-4e19-be74-0f5d7ca0f44d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:07.381617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:07.382608 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:07.382913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:07.383265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:07.385308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:07.505680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:07.510179 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:07.534441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1262/2519] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:34:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:34:07.861633 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668ffb-0105-49b0-90ec-5b08d99d6fac None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:07.862175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668ffb-0105-49b0-90ec-5b08d99d6fac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:07.862841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668ffb-0105-49b0-90ec-5b08d99d6fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:07.903657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a8668ffb-0105-49b0-90ec-5b08d99d6fac None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:07.906825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a8668ffb-0105-49b0-90ec-5b08d99d6fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:07.917640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:07.971004 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1258/2520] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:08.018825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:08.020325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:08.132131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088748a9-f9d9-4e76-ac13-06ed1be1203f service glance] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:08.137840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:08.138504 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:08.138833 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:08.139118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:08.149393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:08.151124 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00745-3284-4888-bc73-b87be6d361bf service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:08.161291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1263/2521] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:18.159996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=68 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:18.229683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=20 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:20.056502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:20.057146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:20.057146 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:20.074591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] MFA Rules not processed for user `73045af76072450a999de6d3fc5962e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:20.075833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:20.083722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1259/2522] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:24.457274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbbe39-c888-4561-9bf2-d1e1cab7cfef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:24.457842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbbe39-c888-4561-9bf2-d1e1cab7cfef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:24.458023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbbe39-c888-4561-9bf2-d1e1cab7cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:24.482018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d3bbbe39-c888-4561-9bf2-d1e1cab7cfef None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:24.482998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d3bbbe39-c888-4561-9bf2-d1e1cab7cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:24.491445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1264/2523] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:34:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:24.507730 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:24.526483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b735b3-e492-4070-aba8-2813edf9577c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:24.534250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:24.535238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:24.535671 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:24.536100 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:24.548160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:24.549789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:24.559182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1260/2524] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:34.476157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:34.555341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=9 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:50.669545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ad26a-4c89-459d-950b-17f6ee96d5f2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:50.670085 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ad26a-4c89-459d-950b-17f6ee96d5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:50.670214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ad26a-4c89-459d-950b-17f6ee96d5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:50.689270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1f3ad26a-4c89-459d-950b-17f6ee96d5f2 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:50.690647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1f3ad26a-4c89-459d-950b-17f6ee96d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:50.698276 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1265/2525] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:50.710739 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:50.722035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082077-87ca-4da0-adb6-da50fa3a7fe6 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:50.727837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:50.728628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:50.728976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:50.729316 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:50.739083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:50.741581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:50.750396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1261/2526] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:51.700955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91576f7-b867-4df7-ab92-da0e93f6c678 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:51.701409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91576f7-b867-4df7-ab92-da0e93f6c678 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:51.701522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91576f7-b867-4df7-ab92-da0e93f6c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:51.720897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a91576f7-b867-4df7-ab92-da0e93f6c678 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:51.722500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a91576f7-b867-4df7-ab92-da0e93f6c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:51.729904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1266/2527] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:34:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:51.744253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:51.754771 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b885-4469-403a-998d-54fb9e3acc69 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:51.760443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:51.761089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:51.761380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:51.761679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:51.770106 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:51.771761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947a6c-d0ec-4999-8d60-a03eccde848f service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:51.780346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1262/2528] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:51.977402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:51.977680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:51.977996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:51.996903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] MFA Rules not processed for user `be6cba32a31a425e964d7222c3d6b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:51.998167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:52.077463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1267/2529] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:52.655690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd88-ce0e-4998-bf11-6e73e83ea213 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:52.656084 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd88-ce0e-4998-bf11-6e73e83ea213 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:52.656435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd88-ce0e-4998-bf11-6e73e83ea213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:52.679461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-83ecdd88-ce0e-4998-bf11-6e73e83ea213 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:52.679461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-83ecdd88-ce0e-4998-bf11-6e73e83ea213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:52.683457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1263/2530] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:34:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:52.696791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:52.711976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c51879-2bee-4b9b-affe-aceb077f5a22 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:52.717184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:52.717886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:52.718181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:52.718464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:52.730474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:52.732131 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:52.740745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1268/2531] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:53.418175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:53.418175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:53.418706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:53.437211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:53.438437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:53.446956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1264/2532] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:53.460170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:53.471125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0c568-777b-476b-a560-8aab53b5e873 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:53.476646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:53.477383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:53.477680 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:53.478002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:53.491863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:53.493488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afcfab-fbb1-4850-9acd-f8dba0e2a91c service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:53.505286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1269/2533] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:34:56.555263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:56.621707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8540-0502-4923-84cc-fb1e9c68a2a5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:56.629132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:56.630057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:56.630347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:56.631182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] PATH_INFO: `/v3/users/4108ab2452684c819e93dabd4f9e7c98` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:56.632413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4108ab2452684c819e93dabd4f9e7c98)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:56.634400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:56.885440 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4108ab2452684c819e93dabd4f9e7c98'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:56.897214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4108ab2452684c819e93dabd4f9e7c98'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:56.985507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4108ab2452684c819e93dabd4f9e7c98'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:56.986185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4108ab2452684c819e93dabd4f9e7c98'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:56.986789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4108ab2452684c819e93dabd4f9e7c98'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.000951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1265/2534] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:34:56 2022] DELETE /identity/v3/users/4108ab2452684c819e93dabd4f9e7c98 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:57.006381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:57.006913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:57.007008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:57.025296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:57.026669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:57.097965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1270/2535] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:57.108829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:57.112709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:57.185096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:57.200237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e329a-655c-494a-8afa-630e1307450a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:57.204485 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:57.205148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:57.205439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:57.205757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:57.216460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:57.218086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c779d9b-5c4d-4760-b813-effc2d0803d1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:57.229588 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1266/2536] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:34:57.558784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:57.561881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:57.609847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6730e7-985d-4e9e-bfc5-70fc13cfc9f7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:57.616533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:57.617516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:57.617827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:57.618123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] PATH_INFO: `/v3/projects/50add55111c4427d94f06a3cf0555067` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:57.619911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50add55111c4427d94f06a3cf0555067)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:57.621903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:57.782242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50add55111c4427d94f06a3cf0555067', 'request_id': 'req-4f13b89f-1f6e-45a2-8e40-8333ba44d944'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.789400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50add55111c4427d94f06a3cf0555067', 'request_id': 'req-4f13b89f-1f6e-45a2-8e40-8333ba44d944'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.866227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50add55111c4427d94f06a3cf0555067', 'request_id': 'req-4f13b89f-1f6e-45a2-8e40-8333ba44d944'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.869443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:57.880818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] The token cache is being invalidate because project 50add55111c4427d94f06a3cf0555067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:34:57.881393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.882400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4f13b89f-1f6e-45a2-8e40-8333ba44d944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:34:57.884120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1271/2537] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:34:57 2022] DELETE /identity/v3/projects/50add55111c4427d94f06a3cf0555067 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:34:57.889259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:57.889761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:57.889898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:57.908761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:57.910066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:57.995670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1267/2538] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:57.997409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:57.998011 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:57.999313 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:58.024295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:58.028288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:58.036329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1272/2539] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:34:58.041090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:58.051024 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d7873-a89d-47f8-a21d-5e14fe780b4f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:58.056250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:58.062036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:58.062414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:58.062663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:58.063344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:34:58.063687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:58.065324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:58.078546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1268/2540] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:34:58.083022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:58.094291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa7c14-dd9e-4f76-966d-b671d8eb8e2f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:58.100060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:58.100060 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:58.100433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:58.100433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:58.101267 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:58.103329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:58.202881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1273/2541] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:34:58 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:34:58.204053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:58.215691 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020e22-66cb-4dad-a35b-bc551cc83c7f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:58.284654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:58.285383 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:58.285672 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:58.286001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:58.309688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:58.309688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:59.271811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1269/2542] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:34:58 2022] POST /identity/v3/users => generated 300 bytes in 1068 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:34:59.274118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:59.287224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257fc5-48d1-4bfe-94a2-c37b2bfda234 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:59.292247 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:59.293350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:59.293485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:59.293724 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:59.294812 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:59.296470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:59.308193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1274/2543] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:34:59.312197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:59.323747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7e63f-408a-4a3d-996a-14a3da097b5a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:59.329318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:59.329318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:59.329746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:59.330073 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:59.330811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:59.332439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c9eaa-6f5d-4a2b-bd8a-96c79d3dc5d3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:59.343029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1270/2544] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:34:59.347320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:59.357914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16c766-2d20-4019-975d-ffea8b1a7074 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:59.363130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:59.364034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:59.364318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:59.364663 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] PATH_INFO: `/v3/projects/34c6c293f12945659ffdb7a0ee4fed0b/users/0842459f23bb491d8a07cd0d8cfffc90/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:59.381381 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c6c293f12945659ffdb7a0ee4fed0b, user_id=0842459f23bb491d8a07cd0d8cfffc90, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:59.384093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:59.527349 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1275/2545] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:34:59 2022] PUT /identity/v3/projects/34c6c293f12945659ffdb7a0ee4fed0b/users/0842459f23bb491d8a07cd0d8cfffc90/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:34:59.595392 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:59.595690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:59.596000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:59.626291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] MFA Rules not processed for user `0842459f23bb491d8a07cd0d8cfffc90`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:34:59.627543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:59.674367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:34:59.703946 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1271/2546] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:34:59.715472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:34:59.717363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:34:59.782357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7910-c89b-46e6-9360-293d7236198d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:34:59.786358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:34:59.787573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:34:59.787862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:34:59.788170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:34:59.797998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:34:59.799749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:34:59.811785 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1276/2547] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:35:08.869813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:08.871336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:08.929890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37cd2-b782-4afa-874c-4076fb35c09e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:08.933481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:08.934152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:08.934437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:08.934714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:08.936610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:08.999945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:09.001446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf51e-1d4f-4960-97e4-79295f79377d service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:09.012332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1272/2548] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:35:09.806493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:12.334258 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:12.401221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24c12d-d44b-48ba-879a-9aadbae84ab5 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:12.407173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:12.408136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:12.408427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:12.408708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] PATH_INFO: `/v3/users/0842459f23bb491d8a07cd0d8cfffc90` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:12.410015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0842459f23bb491d8a07cd0d8cfffc90)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:12.411940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:12.667563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0842459f23bb491d8a07cd0d8cfffc90'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:12.668219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0842459f23bb491d8a07cd0d8cfffc90'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:12.692754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b622-3ac6-47fa-8129-37efa940c558 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:12.693311 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b622-3ac6-47fa-8129-37efa940c558 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:12.693427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b622-3ac6-47fa-8129-37efa940c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:12.710745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-37a3b622-3ac6-47fa-8129-37efa940c558 None None] MFA Rules not processed for user `a1336e374bb542b7a036e3c0d0af54f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:35:12.711968 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37a3b622-3ac6-47fa-8129-37efa940c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:12.768228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0842459f23bb491d8a07cd0d8cfffc90'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:12.769035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0842459f23bb491d8a07cd0d8cfffc90'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:12.771586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1273/2549] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:35:12.777181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0842459f23bb491d8a07cd0d8cfffc90'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:12.797101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1277/2550] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:35:12 2022] DELETE /identity/v3/users/0842459f23bb491d8a07cd0d8cfffc90 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:35:12.801558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:12.801954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:12.803281 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:12.822055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:35:12.823501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:12.852582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:12.884424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1274/2551] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:35:12.894101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:12.899270 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:12.960149 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc14f-7289-40a1-8ad8-9baaed1cc91e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:12.967261 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:12.967958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:12.968242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:12.968518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:12.979549 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:12.981248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e443f-63b4-4262-9b30-b07361f380c0 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:12.992312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1278/2552] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:35:13.650353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:13.651442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:13.666524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a190fb9-4498-4649-a04b-0d33f78be6fa None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:13.671426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:13.672413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:13.672824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:13.672960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] PATH_INFO: `/v3/projects/34c6c293f12945659ffdb7a0ee4fed0b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:13.674210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c6c293f12945659ffdb7a0ee4fed0b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:13.676156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:13.879036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c6c293f12945659ffdb7a0ee4fed0b', 'request_id': 'req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:13.885373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c6c293f12945659ffdb7a0ee4fed0b', 'request_id': 'req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:13.891257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c6c293f12945659ffdb7a0ee4fed0b', 'request_id': 'req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:14.012892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] The token cache is being invalidate because project 34c6c293f12945659ffdb7a0ee4fed0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:35:14.013229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:14.014003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:35:14.015824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1275/2553] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:35:13 2022] DELETE /identity/v3/projects/34c6c293f12945659ffdb7a0ee4fed0b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:35:22.987380 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:24.009221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:38.512191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:38.512701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:38.512799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:38.532192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] MFA Rules not processed for user `bc26c3a3096b4b318d18d4c919f7a60a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:35:38.542538 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:38.618433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1279/2554] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:35:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:35:39.364252 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:39.366648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:39.438571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe756a-b091-4bc2-87f6-88b6ebc6e8b0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:39.442749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:39.443397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:39.443713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:39.443954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:39.445941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:39.537289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:39.538890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:39.551076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1276/2555] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:35:39.616880 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:39.626675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5416fd9-4fd2-471d-adec-101091bdd813 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:39.630379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:39.631084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:39.633285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:39.633285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:39.634353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:39.651506 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:39.653642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:39.685012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1280/2556] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:35:49.546085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=54 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:49.676876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=31 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:35:51.201391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:51.210172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8318-7b83-4278-b9ca-aa64fb2495c9 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:51.214517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:51.215189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:51.215479 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:51.215761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:51.217555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:51.276600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:51.278081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:51.287272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1277/2557] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:35:55.471753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:35:55.474029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:55.489574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be2129-d8c5-4b92-af56-38016194e6c3 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:35:55.496428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:35:55.497166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:35:55.497458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:35:55.497747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:35:55.499822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:35:55.512526 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:35:55.514265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:35:55.524219 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1281/2558] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:01.283030 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=28 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:05.519309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:09.322790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:09.325165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:09.337712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab016e-1da8-4840-8576-68db8f6dca18 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:09.341567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:09.342257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:09.342547 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:09.342831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:09.344715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:09.356164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:09.357892 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:09.366886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1278/2559] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:11.497910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:11.508086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d330b-6f5d-402e-8d6d-eee5e95c29be service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:11.511626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:11.512265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:11.512556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:11.512839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:11.514693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:11.577317 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:11.578766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:11.588364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1282/2560] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:15.638376 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:15.641319 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:15.747433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256b2e-884b-4560-ab88-671efe30d335 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:15.753817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:15.754880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:15.755185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:15.756154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:15.759066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:15.766587 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:15.768099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:15.779254 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:15.781472 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:15.782445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8fb69-45c2-4b11-bbe7-b35014dcc82b service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:15.787572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:15.788603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:15.788959 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:15.789604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] PATH_INFO: `/v3/users/be6cba32a31a425e964d7222c3d6b9fc` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:15.791013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6cba32a31a425e964d7222c3d6b9fc)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:15.793286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:15.794442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1279/2561] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:15.893324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:15.904010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3301a-ce3d-4fcf-bc5c-b5bdb68c36ba service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:15.907620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:15.908265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:15.908555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:15.908836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:15.910673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:15.921051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:15.922510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3191a6-4ab7-4be2-a236-2d9bc5e78177 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:15.931543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1280/2562] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:16.033152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6cba32a31a425e964d7222c3d6b9fc'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.033858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6cba32a31a425e964d7222c3d6b9fc'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.133105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6cba32a31a425e964d7222c3d6b9fc'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.133742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6cba32a31a425e964d7222c3d6b9fc'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.141356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b8e18faa-b44b-4663-9aaa-7acaaecafa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6cba32a31a425e964d7222c3d6b9fc'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.166854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1283/2563] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:36:15 2022] DELETE /identity/v3/users/be6cba32a31a425e964d7222c3d6b9fc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:16.297741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.298326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.298611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.300091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1281/2564] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:36:16.306891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.307691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.308154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.336981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:16.336981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.404431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1284/2565] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:36:16.416970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:16.419069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.432896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf0a21-7a4c-4381-aad5-ace7b07a3392 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:16.436434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:16.437064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.437344 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.437616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.444748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:16.446239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b866eb6-66a6-4573-9875-5105c372dc5b service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:16.457097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1282/2566] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:16.489125 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:16.497359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.511588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:16.551823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053361-c256-4be4-85ed-2ec9057daa71 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.552341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053361-c256-4be4-85ed-2ec9057daa71 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.552745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053361-c256-4be4-85ed-2ec9057daa71 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.555405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1283/2567] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:36:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 14:36:16.560390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.560869 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.561156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.565973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da828-1525-4dab-93a0-4b98ff23faa2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:16.570712 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:16.571656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.571921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.572192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] PATH_INFO: `/v3/projects/e2578b0e75cb455c93cf83dfe54842ff` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.573436 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2578b0e75cb455c93cf83dfe54842ff)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:16.575399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:16.579606 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:16.580823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.648400 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1284/2568] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:36:16.659213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:16.669148 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0a85-904d-4e4e-ac6f-872e973db08e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:16.673053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:16.673709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.674031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.674315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.687438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:16.689068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:16.699751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2578b0e75cb455c93cf83dfe54842ff', 'request_id': 'req-68f3ed10-8083-4645-856e-1622a93f4c57'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.703746 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1285/2569] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:36:16.706296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2578b0e75cb455c93cf83dfe54842ff', 'request_id': 'req-68f3ed10-8083-4645-856e-1622a93f4c57'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.772862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:16.774707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.807824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:16.808951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2578b0e75cb455c93cf83dfe54842ff', 'request_id': 'req-68f3ed10-8083-4645-856e-1622a93f4c57'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.820309 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] The token cache is being invalidate because project e2578b0e75cb455c93cf83dfe54842ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:36:16.823111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.823111 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68f3ed10-8083-4645-856e-1622a93f4c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:16.823495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1285/2570] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:36:16 2022] DELETE /identity/v3/projects/e2578b0e75cb455c93cf83dfe54842ff => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:16.828431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530b7b-57e8-4ff1-8171-7e441d214911 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.828726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530b7b-57e8-4ff1-8171-7e441d214911 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.829014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530b7b-57e8-4ff1-8171-7e441d214911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.845149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e2912-a4c6-4ee1-8c08-ba27559c69a1 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:16.847666 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d5530b7b-57e8-4ff1-8171-7e441d214911 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:16.848879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d5530b7b-57e8-4ff1-8171-7e441d214911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.909416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:16.910463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.910949 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.911399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.914067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.915403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1286/2571] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:36:16.923183 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.923482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.923764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.927861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:16.930960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:16.944960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1286/2572] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:36:16.945518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:16.946869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:16.956238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1287/2573] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:36:16.961184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:16.972033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08379-6492-4aad-909f-79cf4c29a4ad service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:16.977293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:16.978046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:16.978341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:16.978624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:16.979391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:36:16.979964 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:16.981693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:16.997825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1287/2574] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:36:16.999505 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.009844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa034-f304-4479-956e-05565b6ae1aa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.014857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.015589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.015878 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.016161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.016879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.018913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.129475 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1288/2575] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:36:16 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:36:17.133942 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.144262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e8935-fde0-4185-9cf8-970d87aca1d6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.200727 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.201397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.201684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.202002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.202704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.204616 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.287189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1288/2576] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:36:17 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:36:17.291472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.301688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06c7b0-0ea1-4d10-8433-43f32fc40104 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.308379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.309052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.309346 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.309630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.310411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.312037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.328633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1289/2577] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:36:17.336045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.346703 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ed701-936b-4d9c-b7ea-03a3527f6c50 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.352108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.352851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.353363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.353666 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.354506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.357123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052d216-e37b-4b7d-9a38-77b92993bef9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.370187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1289/2578] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:36:17.374370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.385969 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d84ddc-2057-4534-a714-ece9541de09f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.391019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.392029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.392422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.392808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] PATH_INFO: `/v3/projects/7378149db04942bdbc1035f0a3a902fe/users/01c0469b67814077b4544cea54a0192e/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.395626 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:17.410614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7378149db04942bdbc1035f0a3a902fe, user_id=01c0469b67814077b4544cea54a0192e, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.413817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.585593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1290/2579] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:36:17 2022] PUT /identity/v3/projects/7378149db04942bdbc1035f0a3a902fe/users/01c0469b67814077b4544cea54a0192e/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:17.587021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06231a-82f8-4025-8180-7d4db6f9f958 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.587021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06231a-82f8-4025-8180-7d4db6f9f958 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.587198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06231a-82f8-4025-8180-7d4db6f9f958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.607601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7a06231a-82f8-4025-8180-7d4db6f9f958 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:17.609541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7a06231a-82f8-4025-8180-7d4db6f9f958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:17.630813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:17.670563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1290/2580] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:36:17.680832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:17.683274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:17.744476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3665828-307e-458a-9b33-3ebade628986 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:17.748220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:17.748942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:17.749236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:17.749522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:17.759049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:17.760540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:17.770825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1291/2581] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:24.416087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:24.416933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:24.417367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:24.449204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] MFA Rules not processed for user `01c0469b67814077b4544cea54a0192e`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:24.450488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:24.525842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1291/2582] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:36:24.533498 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:24.535135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:24.609158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ea8b-f6e9-43fa-a8e6-176f3d9af181 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:24.614829 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:24.615477 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:24.615793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:24.616068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:24.624065 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:24.625662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:24.635195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1292/2583] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:24.924752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:24.926844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:25.021094 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dbbd0-af59-4c08-9eda-a276b7c0b119 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:25.027202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:25.027939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:25.028280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:25.028488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:25.030369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:25.047468 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:25.048846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:25.050597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:25.051302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:25.085908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1292/2584] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:25.086480 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f610-30e7-4d89-a9cb-5e54a9bf93a5 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:25.101881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:25.102089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:25.102448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:25.103165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] PATH_INFO: `/v3/users/a1336e374bb542b7a036e3c0d0af54f3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:25.104216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1336e374bb542b7a036e3c0d0af54f3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:25.106599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:25.337451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1336e374bb542b7a036e3c0d0af54f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:25.338233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1336e374bb542b7a036e3c0d0af54f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:25.401893 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1336e374bb542b7a036e3c0d0af54f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:25.402611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1336e374bb542b7a036e3c0d0af54f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:25.410485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-83274e2a-4e84-49da-8981-b6f3cb7d7180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1336e374bb542b7a036e3c0d0af54f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:25.413385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:25.432642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1293/2585] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:36:25 2022] DELETE /identity/v3/users/a1336e374bb542b7a036e3c0d0af54f3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:25.822469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:25.835931 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2fa5-cea1-47e5-9ef7-a8ee9615441f service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:25.894368 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:25.909899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:25.910856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:25.911151 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:25.911439 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] PATH_INFO: `/v3/projects/ceeb3ddff16645c6bd9ca8e138ed356f` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:25.912765 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceeb3ddff16645c6bd9ca8e138ed356f)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:25.914676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:26.098980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceeb3ddff16645c6bd9ca8e138ed356f', 'request_id': 'req-af669565-3ec5-4f80-a62d-9247b2df47d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:26.107358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceeb3ddff16645c6bd9ca8e138ed356f', 'request_id': 'req-af669565-3ec5-4f80-a62d-9247b2df47d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:26.113974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceeb3ddff16645c6bd9ca8e138ed356f', 'request_id': 'req-af669565-3ec5-4f80-a62d-9247b2df47d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:26.220483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] The token cache is being invalidate because project ceeb3ddff16645c6bd9ca8e138ed356f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:36:26.220837 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:26.221482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:26.223116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1293/2586] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:36:25 2022] DELETE /identity/v3/projects/ceeb3ddff16645c6bd9ca8e138ed356f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:27.432837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:27.433565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:27.433565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:27.454497 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] MFA Rules not processed for user `01c0469b67814077b4544cea54a0192e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:27.455796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:27.559216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1294/2587] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:36:27.924811 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:27.926276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:27.983689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af669565-3ec5-4f80-a62d-9247b2df47d2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:27.987397 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:27.988077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:27.988370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:27.988650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:27.990445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:28.001411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:28.002951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:28.011324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1294/2588] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:36:29.122499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:29.124284 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:29.221452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cd33f-fb6b-423f-8f10-7b733bcf1031 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:29.231522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:29.232222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:29.233062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:29.233398 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:29.236071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:29.259280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:29.263989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738783c-2b48-4af8-9a13-d097b95bbe9a service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:29.288307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1295/2589] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:31.347914 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:31.351058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:31.386150 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d20f07-9046-459b-a0b8-508dbf2e1afb service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:31.393900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:31.394984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:31.395444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:31.396029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:31.398613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:31.413519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:31.420163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:31.429246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1295/2590] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:39.271728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=65,keystone:DELETE=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:40.339163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:40.340195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:40.340195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:40.357883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:36:40.359133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:40.415052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1296/2591] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:36:40.600626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:40.602192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:40.672261 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c02b-4047-46b9-adff-817bf5f570c2 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:40.677341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:40.678056 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:40.678345 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:40.678627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:40.685398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:40.687854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:40.689458 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:40.697470 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1296/2592] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:36:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:36:41.061846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:41.071000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e65184-f485-4b5f-9389-2a0b74f38d9f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:41.077305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:41.078015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:41.078307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:41.078585 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:41.081604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:41.095875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:41.097468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:41.110049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1297/2593] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:36:50.776188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:50.781428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:50.843830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a8d3-59fe-48b5-b799-f4e3166554eb service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:50.848525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:50.849393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:50.849679 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:50.849986 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] PATH_INFO: `/v3/users/73045af76072450a999de6d3fc5962e6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:50.851179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73045af76072450a999de6d3fc5962e6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:50.853734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:51.102712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73045af76072450a999de6d3fc5962e6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.105117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=31 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:51.115157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73045af76072450a999de6d3fc5962e6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.213262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73045af76072450a999de6d3fc5962e6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.214273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73045af76072450a999de6d3fc5962e6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.214910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73045af76072450a999de6d3fc5962e6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.227797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1297/2594] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:36:50 2022] DELETE /identity/v3/users/73045af76072450a999de6d3fc5962e6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:51.508637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:51.532407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff3e9-05ad-44cd-ad37-101f451507b1 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:51.612334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:51.613360 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:51.613654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:51.613971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] PATH_INFO: `/v3/projects/8ae0529df8aa4640806b4e7f3680e87b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:51.615307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae0529df8aa4640806b4e7f3680e87b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:51.617929 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:51.763859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae0529df8aa4640806b4e7f3680e87b', 'request_id': 'req-1d651c62-f388-418c-93b1-c45ac468d67a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.770212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae0529df8aa4640806b4e7f3680e87b', 'request_id': 'req-1d651c62-f388-418c-93b1-c45ac468d67a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.881738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae0529df8aa4640806b4e7f3680e87b', 'request_id': 'req-1d651c62-f388-418c-93b1-c45ac468d67a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.894542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] The token cache is being invalidate because project 8ae0529df8aa4640806b4e7f3680e87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:36:51.894889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.895510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:51.897189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1298/2595] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:36:51 2022] DELETE /identity/v3/projects/8ae0529df8aa4640806b4e7f3680e87b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:51.901499 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:51.902997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:51.917138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722ff6e-73db-4526-857e-ebdd1cebb8f5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:51.974680 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:51.975565 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:51.975854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:51.976131 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] PATH_INFO: `/v3/users/88604a38aa3e438ea521ee50393c94b2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:51.978007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88604a38aa3e438ea521ee50393c94b2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:51.980198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:51.998235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:52.268326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88604a38aa3e438ea521ee50393c94b2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:52.283861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88604a38aa3e438ea521ee50393c94b2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:52.431879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88604a38aa3e438ea521ee50393c94b2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:52.431879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88604a38aa3e438ea521ee50393c94b2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:52.431879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88604a38aa3e438ea521ee50393c94b2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:52.435733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1298/2596] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:36:51 2022] DELETE /identity/v3/users/88604a38aa3e438ea521ee50393c94b2 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:36:52.775162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:36:52.776687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:36:52.789988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651c62-f388-418c-93b1-c45ac468d67a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:36:52.847723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:36:52.848593 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:36:52.848883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:36:52.849165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] PATH_INFO: `/v3/projects/5618969e1eb64ef3af8d03b451da171f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:36:52.850392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5618969e1eb64ef3af8d03b451da171f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:36:52.852566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:36:52.982687 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:36:52.994825 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5618969e1eb64ef3af8d03b451da171f', 'request_id': 'req-23c461e9-d498-49be-83ce-9a8cec1d7d30'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:53.001192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5618969e1eb64ef3af8d03b451da171f', 'request_id': 'req-23c461e9-d498-49be-83ce-9a8cec1d7d30'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:53.178953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5618969e1eb64ef3af8d03b451da171f', 'request_id': 'req-23c461e9-d498-49be-83ce-9a8cec1d7d30'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:53.191310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] The token cache is being invalidate because project 5618969e1eb64ef3af8d03b451da171f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:36:53.191609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:53.192266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:36:53.193913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1299/2597] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:36:52 2022] DELETE /identity/v3/projects/5618969e1eb64ef3af8d03b451da171f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:37:02.430796 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:03.339328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:17.366369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:17.367828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:17.424051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03dcaa-3981-458c-ad70-e79f6ac2a107 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:17.470911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:17.471567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:17.471879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:17.472158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:17.473999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:17.537343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:17.538852 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:17.548652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1299/2598] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:37:21.135846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:21.137296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:21.151250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c461e9-d498-49be-83ce-9a8cec1d7d30 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:21.156054 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:21.156932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:21.157220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:21.157501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] PATH_INFO: `/v3/users/bc26c3a3096b4b318d18d4c919f7a60a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:21.158748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc26c3a3096b4b318d18d4c919f7a60a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:21.160681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:21.316068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc26c3a3096b4b318d18d4c919f7a60a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.316720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc26c3a3096b4b318d18d4c919f7a60a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.378586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc26c3a3096b4b318d18d4c919f7a60a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.379236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc26c3a3096b4b318d18d4c919f7a60a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.387556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e12fb0b9-b284-4a9d-ae7b-571adeb50b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc26c3a3096b4b318d18d4c919f7a60a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.411426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1300/2599] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:37:21 2022] DELETE /identity/v3/users/bc26c3a3096b4b318d18d4c919f7a60a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:37:21.660937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:21.671758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c260-d7e4-4cb3-a4f6-a865b83c0a4c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:21.736738 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:21.737597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:21.737928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:21.738213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] PATH_INFO: `/v3/projects/9f707e1171364031b016d16a10efda78` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:21.739408 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f707e1171364031b016d16a10efda78)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:21.741309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:21.749958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:21.845881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f707e1171364031b016d16a10efda78', 'request_id': 'req-9d91edcc-8a22-40de-81af-08f9ca882e20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.855287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f707e1171364031b016d16a10efda78', 'request_id': 'req-9d91edcc-8a22-40de-81af-08f9ca882e20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.861195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f707e1171364031b016d16a10efda78', 'request_id': 'req-9d91edcc-8a22-40de-81af-08f9ca882e20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.954728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] The token cache is being invalidate because project 9f707e1171364031b016d16a10efda78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:37:21.955017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.955634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d91edcc-8a22-40de-81af-08f9ca882e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:37:21.957196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1300/2600] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:37:21 2022] DELETE /identity/v3/projects/9f707e1171364031b016d16a10efda78 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:37:21.962236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:21.962487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:21.962761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:21.982861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:21.986589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:22.041937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1301/2601] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:22.049701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.049995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.050280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.082343 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:22.082646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:22.092354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1301/2602] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:22.096630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.110931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0529c-5eae-421e-90fb-5819afd29817 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.115816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.116486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.116846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.117058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.117821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:37:22.118136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.120327 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.132875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1302/2603] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:37:22.137231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.146570 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b21c63-9275-4bf9-9fd1-10eccf8b10e7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.153809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.154488 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.154780 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.155062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.155764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.157676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.249137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1302/2604] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:37:22 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:37:22.253456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.263799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bd17-c9dc-4fe1-9541-6d2fdd1d4959 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.324020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.324697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.324988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.325275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.326024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.327945 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.334656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:22.430098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1303/2605] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:37:22 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:37:22.434331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.444502 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad5a9-1bca-43c2-b5e9-c45d6281edc3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.449115 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.449827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.450155 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.450438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.451168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.452766 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.463650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1303/2606] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:37:22.467627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.477732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5a73-0792-4190-b6e6-26cfb5eece6c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.483431 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.484121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.484410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.484692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.485429 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.487074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180e413-a691-4bf4-9d3f-099c4e8c9079 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.500751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1304/2607] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:37:22.502182 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.512058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b68f-d617-4f08-8b23-34d66d2136a6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.516819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.517686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.518066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.518336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] PATH_INFO: `/v3/projects/abb12869989f45cc875ecb36dd1f7aef/users/98f00ffeef484dd489ed67b96d1ed783/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.533256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb12869989f45cc875ecb36dd1f7aef, user_id=98f00ffeef484dd489ed67b96d1ed783, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.541145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.630761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1304/2608] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:37:22 2022] PUT /identity/v3/projects/abb12869989f45cc875ecb36dd1f7aef/users/98f00ffeef484dd489ed67b96d1ed783/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:37:22.636246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa17fa-9127-4ae8-86eb-e0b6d6264b92 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.636536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa17fa-9127-4ae8-86eb-e0b6d6264b92 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.636867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa17fa-9127-4ae8-86eb-e0b6d6264b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.654820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-dcfa17fa-9127-4ae8-86eb-e0b6d6264b92 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:22.656049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dcfa17fa-9127-4ae8-86eb-e0b6d6264b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:22.714531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1305/2609] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:22.724229 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:22.726027 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:22.782725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf14e-dc52-4a7c-bd3f-65c2ff3bfa84 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:22.786525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:22.787169 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:22.787457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:22.787737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:22.797189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:22.798651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:22.808154 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1305/2610] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:37:28.656718 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e21c7-6a83-4d26-b990-b5cbd7a1b5ed None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:28.657224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e21c7-6a83-4d26-b990-b5cbd7a1b5ed None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:28.657547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e21c7-6a83-4d26-b990-b5cbd7a1b5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:28.682835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0b4e21c7-6a83-4d26-b990-b5cbd7a1b5ed None None] MFA Rules not processed for user `98f00ffeef484dd489ed67b96d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:28.684144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0b4e21c7-6a83-4d26-b990-b5cbd7a1b5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:28.748140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1306/2611] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:28.762871 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:28.764323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:28.806219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:28.841476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f7304-adc1-417a-9939-c76ccdf4053c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:28.847199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:28.847858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:28.848149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:28.848432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:28.857061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:28.858768 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01768f11-4fbc-442d-a002-2018a0476b29 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:28.867845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1306/2612] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:37:31.656117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:31.656672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:31.656672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:31.675716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] MFA Rules not processed for user `98f00ffeef484dd489ed67b96d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:31.676998 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:31.691534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1307/2613] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:39.014241 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=18 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:41.670214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:42.343227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:42.343720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:42.343828 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:42.361842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:37:42.363183 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:42.420826 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1307/2614] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:37:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:37:42.542084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:42.543500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:42.615224 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b957bd0-1772-4951-9d12-d374f53379ad None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:42.620272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:42.621434 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:42.621721 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:42.622032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:42.630648 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:42.632240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:42.641231 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1308/2615] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:37:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:37:52.413062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=25 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:52.637857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=32 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:37:52.777761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:37:52.787235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2bbb9-cfc0-49c0-bb6e-fb1a49928638 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:37:52.790957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:37:52.791683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:37:52.792063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:37:52.792939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:37:52.794827 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:37:52.863980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:37:52.865347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445f126-6c70-4b46-9048-fb1342867be1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:37:52.875259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1308/2616] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:37:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:38:00.567704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:00.577192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e98c-119a-4bdc-86f6-33b083cacfd3 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:00.580756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:00.581396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:00.581714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:00.582057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:00.583836 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:00.596576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:00.598093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7200105-3bd0-45d5-8019-6f0a34264534 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:00.608682 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1309/2617] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:38:00.959391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:00.960399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:00.960953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:00.965364 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1309/2618] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:38:00.971582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea935-2244-499d-9d69-bbef8acfba28 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:00.972024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea935-2244-499d-9d69-bbef8acfba28 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:00.972744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea935-2244-499d-9d69-bbef8acfba28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:00.995332 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a42ea935-2244-499d-9d69-bbef8acfba28 None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:00.999754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a42ea935-2244-499d-9d69-bbef8acfba28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:01.006689 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1310/2619] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:38:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:38:01.018978 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:01.020667 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:01.086601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b2fb-ae44-4091-8af2-be3a1fdbe596 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:01.090882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:01.091605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.091823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.092101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.103744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:01.105858 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27de4c-61f1-4721-a2df-b4e631c22553 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:01.117447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1310/2620] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:38:01.431481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.431768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.432049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.433979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1311/2621] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:38:01.438783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2438-1896-4db8-b5a6-e234d98912ba None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.439076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2438-1896-4db8-b5a6-e234d98912ba None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.439357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2438-1896-4db8-b5a6-e234d98912ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.457972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ad4c2438-1896-4db8-b5a6-e234d98912ba None None] MFA Rules not processed for user `937f40c46a3a4c97897706bbc3425afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:01.459199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ad4c2438-1896-4db8-b5a6-e234d98912ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:01.465924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1311/2622] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:38:01.478418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:01.480624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:01.493710 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03637908-cc43-44c0-a827-ed5c1a2cfd11 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:01.497883 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:01.498529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.498816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.499094 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.506459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:01.507910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb701c-d6d7-432b-a9f6-6e81a4bb262e service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:01.516575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1312/2623] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:38:01.601490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae67e0-b3cc-4f16-8c74-ff438d49aacb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.601928 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae67e0-b3cc-4f16-8c74-ff438d49aacb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.602367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae67e0-b3cc-4f16-8c74-ff438d49aacb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.605276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1312/2624] 10.222.0.93 () {54 vars in 921 bytes} [Thu Jul 14 14:38:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 14:38:01.611561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6079-8d0f-4f27-873a-047a07fffbdb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.612071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6079-8d0f-4f27-873a-047a07fffbdb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.612234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6079-8d0f-4f27-873a-047a07fffbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.631629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-35cb6079-8d0f-4f27-873a-047a07fffbdb None None] MFA Rules not processed for user `47007139f45e4e55b5a9fa2d2901ef9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:01.632845 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-35cb6079-8d0f-4f27-873a-047a07fffbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:01.643049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1313/2625] 10.222.0.93 () {58 vars in 1018 bytes} [Thu Jul 14 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:38:01.654057 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae67e0-b3cc-4f16-8c74-ff438d49aacb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:01.665287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae67e0-b3cc-4f16-8c74-ff438d49aacb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:01.670008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:01.670644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:01.670929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:01.671205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:01.681443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:01.683454 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:01.694338 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1313/2626] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:38:06.249906 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:06.250603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:06.251455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:06.272117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:06.273509 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:06.281511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1314/2627] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:38:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:38:06.294393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:06.306424 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7415-b8f3-4df3-a037-72f5200b2c47 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:06.312393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:06.319721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:06.319721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:06.319721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:06.322754 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:06.324437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:06.333179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1314/2628] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:38:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:38:09.892963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:09.894562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:09.910330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1883-14c0-4416-a708-1135165cadf2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:09.915534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:09.916471 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:09.916756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:09.917565 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] PATH_INFO: `/v3/users/f86340c271b64c14b30834c51a477ca6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:09.919066 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86340c271b64c14b30834c51a477ca6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:09.921113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:10.179816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86340c271b64c14b30834c51a477ca6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.180617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86340c271b64c14b30834c51a477ca6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.279159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86340c271b64c14b30834c51a477ca6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.279948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86340c271b64c14b30834c51a477ca6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.289239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86340c271b64c14b30834c51a477ca6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.308907 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1315/2629] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:38:09 2022] DELETE /identity/v3/users/f86340c271b64c14b30834c51a477ca6 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:38:10.632065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:10.645820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550af03-83f5-4a1d-96bf-b83812ffe472 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:10.723058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:10.724244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:10.724532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:10.724812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] PATH_INFO: `/v3/projects/6b36d14d5bd24ed39bcb0a32dfa016f7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:10.726087 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b36d14d5bd24ed39bcb0a32dfa016f7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:10.727994 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:10.739801 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:10.868450 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b36d14d5bd24ed39bcb0a32dfa016f7', 'request_id': 'req-68b67f14-b769-4779-ab22-31c6e40343b7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.874982 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b36d14d5bd24ed39bcb0a32dfa016f7', 'request_id': 'req-68b67f14-b769-4779-ab22-31c6e40343b7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.880729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b36d14d5bd24ed39bcb0a32dfa016f7', 'request_id': 'req-68b67f14-b769-4779-ab22-31c6e40343b7'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.961850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] The token cache is being invalidate because project 6b36d14d5bd24ed39bcb0a32dfa016f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:38:10.962149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.962809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:10.964447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1315/2630] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:38:10 2022] DELETE /identity/v3/projects/6b36d14d5bd24ed39bcb0a32dfa016f7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:38:10.968657 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:10.970139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:10.985005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f570e17-8df4-47a3-93e4-a716bea7780b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:11.173385 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:11.174209 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:11.174516 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:11.174782 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] PATH_INFO: `/v3/users/b432d9c4986742f7b44772dafe7f384e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:11.176030 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b432d9c4986742f7b44772dafe7f384e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:11.177994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:11.289673 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:11.481250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b432d9c4986742f7b44772dafe7f384e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:11.482588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b432d9c4986742f7b44772dafe7f384e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:11.567214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b432d9c4986742f7b44772dafe7f384e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:11.568002 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b432d9c4986742f7b44772dafe7f384e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:11.576737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-052ef1da-29ec-4411-93f3-64e445daeeca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b432d9c4986742f7b44772dafe7f384e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:11.597133 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1316/2631] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:38:10 2022] DELETE /identity/v3/users/b432d9c4986742f7b44772dafe7f384e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:38:11.911881 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:11.913782 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:11.929821 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67f14-b769-4779-ab22-31c6e40343b7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:12.012104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:12.012979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:12.013267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:12.013544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] PATH_INFO: `/v3/projects/dae55def91d5421a92acf2b1deef5f21` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:12.014813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae55def91d5421a92acf2b1deef5f21)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:12.017581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:12.129707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae55def91d5421a92acf2b1deef5f21', 'request_id': 'req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:12.136917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae55def91d5421a92acf2b1deef5f21', 'request_id': 'req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:12.142596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae55def91d5421a92acf2b1deef5f21', 'request_id': 'req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:12.247446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] The token cache is being invalidate because project dae55def91d5421a92acf2b1deef5f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:38:12.247882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:12.248414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:38:12.250033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1316/2632] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:38:11 2022] DELETE /identity/v3/projects/dae55def91d5421a92acf2b1deef5f21 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:38:21.593373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=19,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:22.243447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:25.323818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae999c-d152-40d3-a1ab-ecc294a8c14a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:25.324917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae999c-d152-40d3-a1ab-ecc294a8c14a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:25.324917 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae999c-d152-40d3-a1ab-ecc294a8c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:25.342356 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-44ae999c-d152-40d3-a1ab-ecc294a8c14a None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:25.343559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-44ae999c-d152-40d3-a1ab-ecc294a8c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:25.400563 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1317/2633] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:38:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:38:25.412900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:25.414355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:25.485993 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ecfe-9f01-4d79-9e4e-d059bbb940c9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:25.491332 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:25.492069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:25.492461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:25.492817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:25.502267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:25.503854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:25.514110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1317/2634] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:38:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:38:35.088776 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594bf1-c985-46c5-8e2b-55e083150786 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:35.090235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594bf1-c985-46c5-8e2b-55e083150786 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:35.090555 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594bf1-c985-46c5-8e2b-55e083150786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:35.108119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-da594bf1-c985-46c5-8e2b-55e083150786 None None] MFA Rules not processed for user `98f00ffeef484dd489ed67b96d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:38:35.109302 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-da594bf1-c985-46c5-8e2b-55e083150786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:35.163132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1318/2635] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:38:35.510074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=32 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:45.156570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=50 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:38:53.797661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:38:53.799101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:53.940333 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efd1e-e0bb-4c38-a3ec-211b74ff337b service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:38:53.945566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:38:53.946265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:38:53.946554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:38:53.946830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:38:53.949177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:38:53.961055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:38:53.962685 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feecf0a-0818-4849-9b26-5856dc0a70d1 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:38:53.973405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1318/2636] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:38:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:39:03.969023 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=63 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:39:11.654856 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12025a-723f-4d4a-914e-07e48a4c1dcf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:39:11.655391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12025a-723f-4d4a-914e-07e48a4c1dcf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:39:11.655391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12025a-723f-4d4a-914e-07e48a4c1dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:39:11.673463 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2c12025a-723f-4d4a-914e-07e48a4c1dcf None None] MFA Rules not processed for user `01c0469b67814077b4544cea54a0192e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:39:11.674751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c12025a-723f-4d4a-914e-07e48a4c1dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:39:11.729861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1319/2637] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:39:21.722757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=25 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:39:43.116278 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:39:43.116781 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:39:43.116943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:39:43.136983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] MFA Rules not processed for user `98f00ffeef484dd489ed67b96d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:39:43.142694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:39:43.145174 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1319/2638] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:39:53.130404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:40:21.726113 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:21.726897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:21.726897 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:21.764241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:40:21.765433 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:21.779853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1320/2639] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:40:21.985108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:40:21.997466 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c4c5-2466-48ed-acf1-d21b5c844a4a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:40:22.002694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:40:22.003346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:22.003662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:22.003945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:22.012835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:40:22.014457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f35a-3802-43b5-a3d0-f57e9387ec4a service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:40:22.022763 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1320/2640] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:40:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:40:31.774269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=10 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:40:32.019318 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:40:40.316307 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:40:40.317965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:40.396521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5bbfe-901e-4b74-9aa2-a0dda361a38f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:40:40.400681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:40:40.401337 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:40.401622 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:40.401940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:40.403751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:40.488889 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:40:40.490942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:40:40.511974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1321/2641] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:40:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:40:41.091785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:41.095246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:41.095246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:41.110995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:40:41.112211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:41.120095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1321/2642] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:40:41.132012 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:40:41.143368 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7bd2-520c-44d2-a59a-8424a3cdc447 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:40:41.149297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:40:41.149971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:41.150256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:41.150533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:41.158636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:40:41.160200 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f4729-9037-435c-8e4f-a4f027c809d2 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:40:41.171114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1322/2643] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:40:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:40:41.752382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:40:41.772712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc169c0-7a63-44d0-a485-bf4751243d83 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:40:41.778957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:40:41.780011 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:41.780474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:41.780926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:41.783969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:41.807925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:40:41.810501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:40:41.831600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1322/2644] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:40:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:40:51.167722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=78 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:40:51.822033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=9 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:40:57.018661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b39b0-2c2c-4b91-b35d-5913e678a9ad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:57.018661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b39b0-2c2c-4b91-b35d-5913e678a9ad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:57.018661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b39b0-2c2c-4b91-b35d-5913e678a9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:57.036900 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-679b39b0-2c2c-4b91-b35d-5913e678a9ad None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:40:57.038137 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-679b39b0-2c2c-4b91-b35d-5913e678a9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:40:57.045408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1323/2645] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:40:57.058593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:40:57.070867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776e9a-7ccc-4e86-b1cf-adeeb7fde85e service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:40:57.076372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:40:57.078041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:40:57.078041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:40:57.078041 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:40:57.089706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:40:57.091351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:40:57.100907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1323/2646] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:40:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:41:04.129154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:04.129643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:04.129793 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:04.147132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] MFA Rules not processed for user `98f00ffeef484dd489ed67b96d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:04.148323 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:04.155155 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1324/2647] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:07.096289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:09.544179 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:09.545714 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:09.558890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d22c6d-1c85-4149-b031-21c09c3177a4 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:09.562595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:09.563237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:09.563527 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:09.563809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:09.565584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:09.575526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:09.576934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:09.585403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1324/2648] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:14.141973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:19.581962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:30.462798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:30.464216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:30.546960 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8522925-2066-4619-8eb9-94fddeb2dc78 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:30.551389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:30.552216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:30.552528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:30.552902 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:30.556624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:30.571039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:30.572479 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:30.582172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1325/2649] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:33.529337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:33.530992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:33.548044 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc0889-4418-401f-8ad0-8d6b227813d7 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:33.554355 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:33.555019 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:33.555306 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:33.555594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:33.557447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:33.571122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:33.572876 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:33.585371 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1325/2650] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:38.545478 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:38.555548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409f0d8-fae3-4051-9bcc-17729cdc50b3 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:38.559088 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:38.559730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:38.560035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:38.560303 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:38.562134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:38.665162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:38.666640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:38.687846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1326/2651] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:41:42.386308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:42.389539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:42.411296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc332-9e31-4126-a9ba-6ee28701ae55 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:42.417649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:42.419642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:42.421320 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:42.422276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:42.428769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:42.442516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:42.444104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:42.454193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1326/2652] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:42.554536 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:42.556037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:42.569161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb2fb-4ef5-4c53-89e0-21f3ef2f36c9 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:42.574513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:42.575300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:42.575685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:42.576128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] PATH_INFO: `/v3/users/01c0469b67814077b4544cea54a0192e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:42.577453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c0469b67814077b4544cea54a0192e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:42.579678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:42.754373 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:42.756920 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c0469b67814077b4544cea54a0192e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:42.757620 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c0469b67814077b4544cea54a0192e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:42.885376 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c0469b67814077b4544cea54a0192e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:42.886041 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c0469b67814077b4544cea54a0192e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:42.894033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-124f2b62-1850-44e2-b470-57e7ed750b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c0469b67814077b4544cea54a0192e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:42.911747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1327/2653] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:41:42 2022] DELETE /identity/v3/users/01c0469b67814077b4544cea54a0192e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:43.207363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:43.219494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275cc1-003a-4ed2-931a-b62ec06da7b1 service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:43.276831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:43.277747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.278065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.278347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] PATH_INFO: `/v3/projects/7378149db04942bdbc1035f0a3a902fe` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.279613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7378149db04942bdbc1035f0a3a902fe)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:43.281558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:43.399619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7378149db04942bdbc1035f0a3a902fe', 'request_id': 'req-81d0e3d9-acd5-4117-885c-c0cf3858ac27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:43.405729 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7378149db04942bdbc1035f0a3a902fe', 'request_id': 'req-81d0e3d9-acd5-4117-885c-c0cf3858ac27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:43.411719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7378149db04942bdbc1035f0a3a902fe', 'request_id': 'req-81d0e3d9-acd5-4117-885c-c0cf3858ac27'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:43.507997 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] The token cache is being invalidate because project 7378149db04942bdbc1035f0a3a902fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:41:43.508291 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:43.508957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-81d0e3d9-acd5-4117-885c-c0cf3858ac27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:43.510625 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1327/2654] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:41:43 2022] DELETE /identity/v3/projects/7378149db04942bdbc1035f0a3a902fe => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:43.515693 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.516077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.516367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.573511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:43.574745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:43.635809 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1328/2655] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:43.644067 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.644361 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.644642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.663571 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:43.664737 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:43.672985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1328/2656] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:43.677335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:43.687223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5850b1-b465-4e1c-935a-68683bb359ab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:43.691928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:43.692726 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.693014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.693294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.694056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:41:43.694370 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:43.695978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:43.709198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1329/2657] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:41:43.713519 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:43.724105 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45faaa-937a-4033-9af9-23a5bd9ea919 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:43.728778 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:43.729433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.729726 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.730039 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.730747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:43.732704 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:43.830943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1329/2658] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:41:43 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:41:43.835172 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:43.845487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1e94-ed26-497b-be1d-5cced7f89511 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:43.903031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:43.903705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:43.903992 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:43.904272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:43.904976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:43.906927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:43.989472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1330/2659] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:41:43 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:41:43.993657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:44.004117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb19d-56b6-4464-be73-f5c972110522 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:44.008655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:44.009305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:44.009682 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:44.010008 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:44.010735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:44.012323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:44.022236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1330/2660] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:44.028192 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:44.038847 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a785-5921-4f74-8379-2b6ab01fcef3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:44.043681 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:44.044343 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:44.044633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:44.044934 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:44.045646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:44.047266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8b683-1472-4b45-a9fd-9410936de406 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:44.048556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:44.060521 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1331/2661] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:44.064481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:44.074921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb618-08aa-45ac-86ce-ee909e215244 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:44.079719 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:44.082743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:44.082743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:44.082743 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] PATH_INFO: `/v3/projects/f966eb505d494be6814cb3b01157ecee/users/433e5818662a4e2eb3b15954cbc8d612/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:44.093650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:44.097563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f966eb505d494be6814cb3b01157ecee, user_id=433e5818662a4e2eb3b15954cbc8d612, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:44.100223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:44.200469 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1331/2662] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:41:44 2022] PUT /identity/v3/projects/f966eb505d494be6814cb3b01157ecee/users/433e5818662a4e2eb3b15954cbc8d612/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:44.210758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:44.211055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:44.211339 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:44.238205 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] MFA Rules not processed for user `433e5818662a4e2eb3b15954cbc8d612`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:44.239420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:44.303530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1332/2663] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:44.314339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:44.316637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:44.373522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7b8ee-f9ff-4f8c-9fb4-58cd5d076b8a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:44.377053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:44.378298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:44.378582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:44.378861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:44.387255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:44.388677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ee94-70ff-42b5-bda3-161428ab700d service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:44.397396 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1332/2664] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:54.300927 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=34 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:54.393360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:54.941658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:55.003753 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ab265-44f9-4ea2-aeae-143bbcdefa67 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:55.008118 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:55.008983 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:55.009259 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:55.009533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] PATH_INFO: `/v3/users/433e5818662a4e2eb3b15954cbc8d612` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:55.010954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433e5818662a4e2eb3b15954cbc8d612)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:55.012850 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:55.186348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433e5818662a4e2eb3b15954cbc8d612'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:55.187018 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433e5818662a4e2eb3b15954cbc8d612'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:55.262101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433e5818662a4e2eb3b15954cbc8d612'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:55.262678 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433e5818662a4e2eb3b15954cbc8d612'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:55.270564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433e5818662a4e2eb3b15954cbc8d612'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:55.295301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1333/2665] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:41:54 2022] DELETE /identity/v3/users/433e5818662a4e2eb3b15954cbc8d612 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:55.299798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:55.300089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:55.300367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:55.317835 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:55.321611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:55.378555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1333/2666] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:55.388719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:55.390442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:55.448451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f07295-4062-4dd6-af89-dc898f2238e6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:55.452082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:55.452729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:55.453051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:55.453336 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:55.463071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:55.464501 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1ac76-9e50-49a9-8e1e-7b9f8a4f1b2a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:55.474925 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1334/2667] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:55.915141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:55.919082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:55.930012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4ec4-1b71-4a20-bee2-fccbbdd3e246 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:55.934670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:55.935543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:55.936846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:55.936846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] PATH_INFO: `/v3/projects/f966eb505d494be6814cb3b01157ecee` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:55.937546 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f966eb505d494be6814cb3b01157ecee)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:55.939514 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.103838 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f966eb505d494be6814cb3b01157ecee', 'request_id': 'req-9d2835a5-7c26-4818-9ad3-d9b9d8404016'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:56.109943 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f966eb505d494be6814cb3b01157ecee', 'request_id': 'req-9d2835a5-7c26-4818-9ad3-d9b9d8404016'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:56.115512 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f966eb505d494be6814cb3b01157ecee', 'request_id': 'req-9d2835a5-7c26-4818-9ad3-d9b9d8404016'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:56.205969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] The token cache is being invalidate because project f966eb505d494be6814cb3b01157ecee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:41:56.206227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:56.206798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9d2835a5-7c26-4818-9ad3-d9b9d8404016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:56.208761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1334/2668] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:41:55 2022] DELETE /identity/v3/projects/f966eb505d494be6814cb3b01157ecee => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:56.213738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.214168 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.214570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.232958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:56.234191 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:56.250490 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:56.292488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1335/2669] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:56.301705 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.302065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.302380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.320839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:56.322081 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:56.330908 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1335/2670] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:41:56.335086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.344704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4256-b679-4eab-842e-2e441114a1f4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.350745 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.351401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.351683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.351958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.352688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:41:56.353023 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.354784 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.366075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1336/2671] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:41:56.370394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.380088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e7cf6-e0a8-418b-8164-8e587a2e53be None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.384741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.385389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.385673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.385981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.386684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.388578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.472854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1336/2672] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:41:56.479084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.484397 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5ca1-4b6a-41c5-bd88-997c1745e7e4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.541971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.542642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.542926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.543203 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.543905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.545810 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.644545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1337/2673] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:41:56.649157 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.659802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d39f5-825e-4ef6-b998-1c7d1e4f5252 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.664537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.665211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.665536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.665888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.666646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.668244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.680156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1337/2674] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:56.684099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.695163 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f709-21d1-4de1-89a2-d2c404e1b46a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.699855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.700690 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.700978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.701256 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.702028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.703598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de381b66-9451-4b7b-a6ad-48b08336fc31 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.714413 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1338/2675] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:56.718652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.728764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1d153-af54-4c7c-9d2c-4b36f6d36431 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:56.733416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:56.734323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.734614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.734898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] PATH_INFO: `/v3/projects/47ac60ab65554a3cb42de19160e806b2/users/ce861184188e45b1a3dd7bd2ec08347e/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.749830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ac60ab65554a3cb42de19160e806b2, user_id=ce861184188e45b1a3dd7bd2ec08347e, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:56.753143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:56.851511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1338/2676] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:41:56 2022] PUT /identity/v3/projects/47ac60ab65554a3cb42de19160e806b2/users/ce861184188e45b1a3dd7bd2ec08347e/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:56.862640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:56.865918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:56.865918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:56.889979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] MFA Rules not processed for user `ce861184188e45b1a3dd7bd2ec08347e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:56.891173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:56.954354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1339/2677] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:56.959461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:56.982099 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:57.023097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dca77d-97b8-440f-af57-4075c19fac8a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.027779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.028432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.028720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.029001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.029689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.031622 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.144706 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1339/2678] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:41:56 2022] POST /identity/v3/projects => generated 363 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:41:57.148853 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.159918 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55fe33-1491-49d9-b0f4-3bbaabe622e8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.172126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:57.218310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.218990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.219341 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.219632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.220965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.223186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.308515 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1340/2679] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:41:57 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:41:57.312711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.323236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac172d4-d74c-4485-9d3b-f4753cc9b98c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.327922 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.328582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.328874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.329160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.329926 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.331526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.342194 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1340/2680] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:57.347120 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.357329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2b6ef-c0b1-4a5f-89f1-4c2dc5d4a2d4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.362605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.363485 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.363773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.364055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] PATH_INFO: `/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7/users/0043898334314f9abad22a2c6ab60624/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.379014 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3ec25ffdf4995a4587203e46a3fc7, user_id=0043898334314f9abad22a2c6ab60624, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.381692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.504010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1341/2681] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:41:57 2022] PUT /identity/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7/users/0043898334314f9abad22a2c6ab60624/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:57.507969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.517903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9cf2-f5a6-4bef-95f0-4adc97e5cf80 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.574196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.574787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.575101 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.575385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.576122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.577715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56608e2e-f356-4e47-8dfe-6ee661952e5a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.587448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1341/2682] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:41:57.591688 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.602358 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205101ef-db79-4f14-b591-f6474f6e6ce2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.607116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.608035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.608320 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.608599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] PATH_INFO: `/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7/users/0043898334314f9abad22a2c6ab60624/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.610637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3ec25ffdf4995a4587203e46a3fc7, user_id=0043898334314f9abad22a2c6ab60624, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.613787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.624033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1342/2683] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:41:57 2022] PUT /identity/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7/users/0043898334314f9abad22a2c6ab60624/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:57.635925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f680-39b4-4382-be42-e69f219e057a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.636196 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f680-39b4-4382-be42-e69f219e057a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.636476 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f680-39b4-4382-be42-e69f219e057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.662509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c942f680-39b4-4382-be42-e69f219e057a None None] MFA Rules not processed for user `0043898334314f9abad22a2c6ab60624`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:57.663733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c942f680-39b4-4382-be42-e69f219e057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:57.731221 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1342/2684] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:57.743045 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:57.744933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:57.803061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746672-e287-47ee-8b66-79ad6c38b6c3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:57.806733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:57.807383 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:57.807675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:57.807958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:57.817547 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:57.819013 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777a808-46f8-44f9-a9df-3e152239b1d8 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:57.828562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1343/2685] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:58.668078 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f680-39b4-4382-be42-e69f219e057a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:58.681097 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:58.724937 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f680-39b4-4382-be42-e69f219e057a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:58.729500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:58.730403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:58.730693 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:58.730972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] PATH_INFO: `/v3/users/ce861184188e45b1a3dd7bd2ec08347e` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:58.732210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce861184188e45b1a3dd7bd2ec08347e)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:58.734189 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:58.938253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce861184188e45b1a3dd7bd2ec08347e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:58.950991 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce861184188e45b1a3dd7bd2ec08347e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.016595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce861184188e45b1a3dd7bd2ec08347e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.017201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce861184188e45b1a3dd7bd2ec08347e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.017824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce861184188e45b1a3dd7bd2ec08347e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.030510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1343/2686] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:41:58 2022] DELETE /identity/v3/users/ce861184188e45b1a3dd7bd2ec08347e => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:41:59.035069 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:59.035558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:59.035674 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:59.052862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:41:59.054112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:59.114467 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1344/2687] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:41:59.124223 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:59.125649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:59.182230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1bcc-6817-4ac6-8b87-462d96701a46 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:59.185904 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:59.186551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:59.186841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:59.187123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:59.196639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:59.198127 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:59.207544 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1344/2688] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:41:59.652129 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:59.653624 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:41:59.666483 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617e8f-cd0e-4f6e-9f8f-0aebf428a342 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:59.672800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:59.674301 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:59.674764 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:59.675096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] PATH_INFO: `/v3/projects/47ac60ab65554a3cb42de19160e806b2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:59.676345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ac60ab65554a3cb42de19160e806b2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:59.678305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:41:59.687074 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:59.821453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ac60ab65554a3cb42de19160e806b2', 'request_id': 'req-b201620b-ab1f-486d-847d-165a2e1f6bf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.827924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ac60ab65554a3cb42de19160e806b2', 'request_id': 'req-b201620b-ab1f-486d-847d-165a2e1f6bf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.902350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ac60ab65554a3cb42de19160e806b2', 'request_id': 'req-b201620b-ab1f-486d-847d-165a2e1f6bf3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.913862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] The token cache is being invalidate because project 47ac60ab65554a3cb42de19160e806b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:41:59.914158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.914801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:41:59.916340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1345/2689] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:41:59 2022] DELETE /identity/v3/projects/47ac60ab65554a3cb42de19160e806b2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:41:59.920553 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:41:59.930326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f3d8-92c1-4b02-aeb0-8a3bc79654be service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:41:59.966297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:41:59.989976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:41:59.990847 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:41:59.991140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:41:59.991421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] PATH_INFO: `/v3/users/0043898334314f9abad22a2c6ab60624` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:41:59.992711 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0043898334314f9abad22a2c6ab60624)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:41:59.994686 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:00.218507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0043898334314f9abad22a2c6ab60624'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:00.233405 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0043898334314f9abad22a2c6ab60624'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:00.319555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0043898334314f9abad22a2c6ab60624'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:00.320188 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0043898334314f9abad22a2c6ab60624'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:00.320797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a6004d90-77c8-49f6-88f0-2b09665671a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0043898334314f9abad22a2c6ab60624'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:00.333861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1345/2690] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:41:59 2022] DELETE /identity/v3/users/0043898334314f9abad22a2c6ab60624 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:00.778564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:00.780662 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:00.805147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201620b-ab1f-486d-847d-165a2e1f6bf3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:00.872318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:00.873508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:00.874099 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:00.874613 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] PATH_INFO: `/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:00.877427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b3ec25ffdf4995a4587203e46a3fc7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:00.879818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.002805 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b3ec25ffdf4995a4587203e46a3fc7', 'request_id': 'req-dd16ac2e-ddac-4e42-853f-a6424b334a95'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:01.009199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b3ec25ffdf4995a4587203e46a3fc7', 'request_id': 'req-dd16ac2e-ddac-4e42-853f-a6424b334a95'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:01.108234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b3ec25ffdf4995a4587203e46a3fc7', 'request_id': 'req-dd16ac2e-ddac-4e42-853f-a6424b334a95'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:01.119552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] The token cache is being invalidate because project d0b3ec25ffdf4995a4587203e46a3fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:01.119846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:01.120487 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dd16ac2e-ddac-4e42-853f-a6424b334a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:01.122044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1346/2691] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:00 2022] DELETE /identity/v3/projects/d0b3ec25ffdf4995a4587203e46a3fc7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:01.127543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.127999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.128096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.147862 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:01.149065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:01.217425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1346/2692] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:01.219374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.219552 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.219806 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.239514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:01.240730 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:01.249645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1347/2693] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:01.254885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.264678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579e351-44a6-4f13-8f8b-02b144aa6a93 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.269267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.270029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.270324 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.270607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.271337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:42:01.271648 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.273240 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.284877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1347/2694] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:42:01.289236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.299363 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed828720-8dc1-42c9-b146-3758285b4827 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.304090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.304755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.305092 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.305400 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.306140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.308213 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.405401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:01.412835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1348/2695] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:01.416962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.427294 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03ecd-7305-4b84-9923-b51182e7bf48 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.469231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:01.483960 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.484626 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.484917 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.485197 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.486443 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.488351 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.564872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1348/2696] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:01.568921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.580402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0a3-b470-4feb-9ce0-59503daef4cc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.584988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.585642 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.585957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.586239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.586962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.588638 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.599931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1349/2697] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:01.603920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.613948 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53906c8-98c2-44cb-a0f3-dc17e71f9dfb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.618582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.619246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.619639 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.619856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.620713 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.622425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11737d8c-d02f-4676-bb3c-02f7d58b4aac admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.633867 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1349/2698] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:01.637750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.648868 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cd434-2d87-47f9-84db-f54fcf2b0ef7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.654884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.655773 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.656056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.656330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] PATH_INFO: `/v3/projects/72b5582914cc47a6be0916145fe95da6/users/c4acf161aecb44aaa3ed7a31419fc9ab/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.671822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b5582914cc47a6be0916145fe95da6, user_id=c4acf161aecb44aaa3ed7a31419fc9ab, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.674566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:01.776832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1350/2699] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:01 2022] PUT /identity/v3/projects/72b5582914cc47a6be0916145fe95da6/users/c4acf161aecb44aaa3ed7a31419fc9ab/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:01.787051 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.792202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.792202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.813589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] MFA Rules not processed for user `c4acf161aecb44aaa3ed7a31419fc9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:01.820697 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:01.880953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1350/2700] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:01.885445 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:01.947330 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0d1f-75c4-4d41-aeca-0a64cca89115 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:01.952148 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:01.952811 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:01.953107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:01.953393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:01.954143 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:01.956598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.070350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1351/2701] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:01 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:02.074749 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.086394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ea576-65d5-43e5-b0e5-dc65da3b5586 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.145093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.145805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.146086 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.146367 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.147071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.148987 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.298695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1351/2702] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:02 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:02.302512 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.315863 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c579767-12ff-45d7-9d80-0c84d0b6a647 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.323348 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.324096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.324482 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.324953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.325854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.327916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.340001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1352/2703] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:42:02.344789 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.356003 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a920-9932-4dbc-9bfb-948f408b2dc7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.361292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.362216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.362507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.362794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] PATH_INFO: `/v3/projects/dc07481e6e7942769e64cb124d814cec/users/72aa2e7de30f4bf48ddafdc6ccbf1746/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.378516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc07481e6e7942769e64cb124d814cec, user_id=72aa2e7de30f4bf48ddafdc6ccbf1746, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.381212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.462817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1352/2704] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:02 2022] PUT /identity/v3/projects/dc07481e6e7942769e64cb124d814cec/users/72aa2e7de30f4bf48ddafdc6ccbf1746/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:02.466818 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.478061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe784-643b-435a-8b0c-fff166ea027f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.533942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.534635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.535007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.535285 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.536008 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.537607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fd7a5-d7e3-4644-97a9-9dcdd9e9beee admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.547766 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1353/2705] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:02.552657 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.563063 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a069b9c-397f-4828-9932-09cc2211b7a8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.567819 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.568689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.568970 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.569250 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] PATH_INFO: `/v3/projects/dc07481e6e7942769e64cb124d814cec/users/72aa2e7de30f4bf48ddafdc6ccbf1746/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.571231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc07481e6e7942769e64cb124d814cec, user_id=72aa2e7de30f4bf48ddafdc6ccbf1746, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.573866 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.584220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1353/2706] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:02 2022] PUT /identity/v3/projects/dc07481e6e7942769e64cb124d814cec/users/72aa2e7de30f4bf48ddafdc6ccbf1746/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:02.647122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.647510 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.649101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.676716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] MFA Rules not processed for user `72aa2e7de30f4bf48ddafdc6ccbf1746`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:02.678115 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:02.698055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:02.749272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1354/2707] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:02.760401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:02.762168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:02.769413 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:02.831069 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b4e2a-9c1d-4db4-b8c4-38d1577ebe53 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:02.834944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:02.835537 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:02.835829 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:02.836111 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:02.847788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:02.849244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8310c3-fdd1-40be-93ea-6b0bc7ee8b02 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:02.859418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1354/2708] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:03.006990 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:03.071700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624e7e6-c599-4d4b-9f79-acf92b05fee2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:03.076098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:03.076955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:03.077239 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:03.077519 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] PATH_INFO: `/v3/users/c4acf161aecb44aaa3ed7a31419fc9ab` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:03.078728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4acf161aecb44aaa3ed7a31419fc9ab)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:03.081167 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:03.263325 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4acf161aecb44aaa3ed7a31419fc9ab'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:03.263974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4acf161aecb44aaa3ed7a31419fc9ab'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:03.346943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4acf161aecb44aaa3ed7a31419fc9ab'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:03.347594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4acf161aecb44aaa3ed7a31419fc9ab'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:03.355347 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4acf161aecb44aaa3ed7a31419fc9ab'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:03.372852 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1355/2709] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:42:03 2022] DELETE /identity/v3/users/c4acf161aecb44aaa3ed7a31419fc9ab => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:03.377932 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:03.378403 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:03.378526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:03.396530 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:03.397747 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:03.456395 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1355/2710] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:03.466424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:03.467866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:03.510112 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:03.525617 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ebec-a183-40b2-b38b-4df8399c0cc7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:03.528362 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:03.529005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:03.529289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:03.529566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:03.538632 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:03.540090 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:03.550646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1356/2711] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:03.994644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:03.996298 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:04.008477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6094c-3dd4-4d57-8d31-70766de0952b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:04.013436 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:04.014341 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:04.015065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:04.015350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] PATH_INFO: `/v3/projects/72b5582914cc47a6be0916145fe95da6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:04.016794 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b5582914cc47a6be0916145fe95da6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:04.019609 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:04.194168 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b5582914cc47a6be0916145fe95da6', 'request_id': 'req-ef739806-413e-46f1-a1b2-a9672a330275'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.200358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b5582914cc47a6be0916145fe95da6', 'request_id': 'req-ef739806-413e-46f1-a1b2-a9672a330275'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.206415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b5582914cc47a6be0916145fe95da6', 'request_id': 'req-ef739806-413e-46f1-a1b2-a9672a330275'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.353089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] The token cache is being invalidate because project 72b5582914cc47a6be0916145fe95da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:04.353538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.354214 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.355850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1356/2712] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:03 2022] DELETE /identity/v3/projects/72b5582914cc47a6be0916145fe95da6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:04.360484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:04.370951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b73d6-fcc7-46a0-8e37-6385a7555a2f service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:04.429891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:04.430777 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:04.431407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:04.431407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] PATH_INFO: `/v3/users/72aa2e7de30f4bf48ddafdc6ccbf1746` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:04.432627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72aa2e7de30f4bf48ddafdc6ccbf1746)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:04.434599 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:04.671443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72aa2e7de30f4bf48ddafdc6ccbf1746'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.672215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72aa2e7de30f4bf48ddafdc6ccbf1746'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.737032 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72aa2e7de30f4bf48ddafdc6ccbf1746'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.737715 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72aa2e7de30f4bf48ddafdc6ccbf1746'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.745443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ce204324-5499-42b6-a9c3-f51515f9eb29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72aa2e7de30f4bf48ddafdc6ccbf1746'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:04.765919 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1357/2713] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:42:04 2022] DELETE /identity/v3/users/72aa2e7de30f4bf48ddafdc6ccbf1746 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:05.471237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:05.472872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:05.487002 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739806-413e-46f1-a1b2-a9672a330275 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:05.518906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:05.548760 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:05.549627 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:05.549953 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:05.550230 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] PATH_INFO: `/v3/projects/dc07481e6e7942769e64cb124d814cec` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:05.551409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc07481e6e7942769e64cb124d814cec)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:05.553387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:05.707812 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc07481e6e7942769e64cb124d814cec', 'request_id': 'req-7102ed26-aa6a-4c94-8740-36f35dd1da80'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:05.714204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc07481e6e7942769e64cb124d814cec', 'request_id': 'req-7102ed26-aa6a-4c94-8740-36f35dd1da80'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:05.719638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc07481e6e7942769e64cb124d814cec', 'request_id': 'req-7102ed26-aa6a-4c94-8740-36f35dd1da80'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:05.811321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] The token cache is being invalidate because project dc07481e6e7942769e64cb124d814cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:05.811617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:05.812257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7102ed26-aa6a-4c94-8740-36f35dd1da80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:05.814038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1357/2714] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:05 2022] DELETE /identity/v3/projects/dc07481e6e7942769e64cb124d814cec => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:05.818940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:05.819462 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:05.819598 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:05.838796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:05.840079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:05.895507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:05.906486 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1358/2715] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:05.915715 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:05.915834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:05.916074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:05.943879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:05.945228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:05.956735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1358/2716] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:05.961169 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:05.973802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c26b5-69b8-4d34-9779-b3cbbc8124ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:05.982511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:05.983899 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:05.986399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:05.986768 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:05.987683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:42:05.989244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:05.991067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.006037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1359/2717] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:42:06.009248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.020300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34fe31-bbc2-479e-b552-551d542b10e8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.025425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.026204 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.026567 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.026913 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.027732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.030486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.157610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1359/2718] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:06 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:42:06.161926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.175933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9d313-1c5f-42a2-87dc-319d59979824 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.262621 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.263306 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.263596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.263884 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.264588 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.267019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.384212 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1360/2719] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:06 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:06.388352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.400366 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae2709-6d7c-478a-9094-3c4de1e9d9a4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.405981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.406370 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.406658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.406938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.408313 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.410009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.439874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1360/2720] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:06.443864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.455280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d75ca5-2ad7-404b-9684-a73e5d0ebf63 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.462236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.462933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.463198 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.463476 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.464210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.465882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da9c0-d070-4f70-8659-1f13c4369e39 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.478000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1361/2721] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:06.484515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.492234 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9847cd-37bf-4846-ad2c-d50d16e084c4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.496936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.497857 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.498185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.498421 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] PATH_INFO: `/v3/projects/35413c62675b457a84da91d845f96f2b/users/336a80f361934b8d8ed214883f652d60/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.514981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35413c62675b457a84da91d845f96f2b, user_id=336a80f361934b8d8ed214883f652d60, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.518193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.628043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1361/2722] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:06 2022] PUT /identity/v3/projects/35413c62675b457a84da91d845f96f2b/users/336a80f361934b8d8ed214883f652d60/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:06.633379 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22e788-b386-452e-862f-3eb3fcf0b06d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.633708 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22e788-b386-452e-862f-3eb3fcf0b06d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.634037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22e788-b386-452e-862f-3eb3fcf0b06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.652685 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5d22e788-b386-452e-862f-3eb3fcf0b06d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:06.653760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5d22e788-b386-452e-862f-3eb3fcf0b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:06.712765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1362/2723] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:06.723651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:06.725410 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:06.781971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee661-d1a1-4459-ae97-2a5340a4231f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:06.786256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:06.786906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:06.787195 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:06.787477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:06.796732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:06.798237 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:06.803181 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:06.807938 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1362/2724] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:08.548831 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:08.549328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:08.549443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:08.580994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] MFA Rules not processed for user `336a80f361934b8d8ed214883f652d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:08.582264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:08.642968 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:08.652916 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1363/2725] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:08.657939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:08.668119 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c3463-df08-4a97-8f0b-f9013831bb32 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:08.672823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:08.674198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:08.674482 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:08.674756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:08.675461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:08.677350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:08.805962 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1363/2726] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:08 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:08.810126 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:08.820701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987cfb4-f20e-4455-9cb2-4caedc92a8c1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:08.880178 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:08.880942 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:08.881240 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:08.881522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:08.882318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:08.884227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:09.006291 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1364/2727] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:08 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:09.010592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:09.021284 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f78-78df-48bb-9679-4fe6eb6bb9fb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:09.025861 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:09.026497 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:09.026788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:09.027072 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:09.027798 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:09.029399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:09.040610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1364/2728] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:42:09.044570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:09.054729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef72e6-8a1f-4ce5-bfac-a1109197d732 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:09.059668 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:09.060556 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:09.060848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:09.061165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] PATH_INFO: `/v3/projects/09b9181d380e44009bc3f01b94dad329/users/13b54bda40e242b0ada8a63bde1d6ed2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:09.076634 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b9181d380e44009bc3f01b94dad329, user_id=13b54bda40e242b0ada8a63bde1d6ed2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:09.079926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:09.170459 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1365/2729] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:09 2022] PUT /identity/v3/projects/09b9181d380e44009bc3f01b94dad329/users/13b54bda40e242b0ada8a63bde1d6ed2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:09.174481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:09.184893 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f88d0-56d0-4af4-915b-01c4f5f66ab4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:09.240596 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:09.241259 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:09.241545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:09.241849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:09.242579 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:09.244172 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a9828-0cf1-468d-8bdf-1834165d9129 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:09.254349 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1365/2730] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:09.258614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:09.269629 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50723390-673a-4b40-a476-14059b1a7374 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:09.274414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:09.275288 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:09.275571 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:09.275848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] PATH_INFO: `/v3/projects/09b9181d380e44009bc3f01b94dad329/users/13b54bda40e242b0ada8a63bde1d6ed2/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:09.277819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b9181d380e44009bc3f01b94dad329, user_id=13b54bda40e242b0ada8a63bde1d6ed2, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:09.280427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:09.290403 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1366/2731] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:09 2022] PUT /identity/v3/projects/09b9181d380e44009bc3f01b94dad329/users/13b54bda40e242b0ada8a63bde1d6ed2/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:10.790676 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:10.791708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:10.791708 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:10.819802 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] MFA Rules not processed for user `13b54bda40e242b0ada8a63bde1d6ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:10.821116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:10.892660 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1366/2732] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:10.903994 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:10.905744 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:10.964359 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822689a-695c-4cc0-8adc-2d3edaeaeb39 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:10.968154 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:10.968832 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:10.969122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:10.969427 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:11.032937 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:11.034414 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:11.043281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1367/2733] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:12.801521 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:12.811620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6cea0-3b68-44e5-acc5-d7b5f5bc87f9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:12.815382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:12.816538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:12.816831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:12.817116 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:12.827879 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:12.829558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:12.840372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1367/2734] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:16.615821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:16.617222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:16.622597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:16.685817 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310c24-a82f-48ad-a271-426e21428dc3 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:16.691791 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:16.692529 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:16.692921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:16.693208 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:16.695513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:16.708837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:16.710508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:16.721528 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1368/2735] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:16.831923 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:16.833420 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:16.875285 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:16.900295 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962f3e-e7b5-48ad-83b8-97de6592fb44 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:16.905012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:16.905929 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:16.906225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:16.906509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] PATH_INFO: `/v3/users/98f00ffeef484dd489ed67b96d1ed783` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:16.907709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f00ffeef484dd489ed67b96d1ed783)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:16.910350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:17.126610 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f00ffeef484dd489ed67b96d1ed783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.138833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f00ffeef484dd489ed67b96d1ed783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.237433 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f00ffeef484dd489ed67b96d1ed783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.238088 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f00ffeef484dd489ed67b96d1ed783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.238689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-823cbd50-bf03-4c9a-aae3-69bca97ba99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f00ffeef484dd489ed67b96d1ed783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.252744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1368/2736] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:42:16 2022] DELETE /identity/v3/users/98f00ffeef484dd489ed67b96d1ed783 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:17.541830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:17.554584 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ce34-dee1-488d-adfa-8beb6394304d service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:17.611322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:17.612104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:17.614290 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:17.614572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] PATH_INFO: `/v3/projects/abb12869989f45cc875ecb36dd1f7aef` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:17.615820 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb12869989f45cc875ecb36dd1f7aef)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:17.617713 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:17.757334 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb12869989f45cc875ecb36dd1f7aef', 'request_id': 'req-5cef5121-cc03-4336-8d0a-11d4f724ed8e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.765237 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb12869989f45cc875ecb36dd1f7aef', 'request_id': 'req-5cef5121-cc03-4336-8d0a-11d4f724ed8e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.860569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb12869989f45cc875ecb36dd1f7aef', 'request_id': 'req-5cef5121-cc03-4336-8d0a-11d4f724ed8e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.873235 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] The token cache is being invalidate because project abb12869989f45cc875ecb36dd1f7aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:17.873484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.874140 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5cef5121-cc03-4336-8d0a-11d4f724ed8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:17.875694 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1369/2737] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:17 2022] DELETE /identity/v3/projects/abb12869989f45cc875ecb36dd1f7aef => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:17.881445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:17.881934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:17.881934 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:17.901356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:17.902615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:17.961809 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1369/2738] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:17.970449 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:17.970755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:17.971037 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:17.989888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:17.991109 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:17.999016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1370/2739] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:42:18.007256 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.016955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f00978-fb6b-49d9-acbd-d33eb3c395db None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.021954 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.022615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.022930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.023211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.023945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:42:18.024255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.025915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.037939 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1370/2740] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:42:18.042365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.052557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a763724-d751-4d87-8874-25aa059b2546 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.057951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.058612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.058898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.059180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.059888 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.061861 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.075719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:18.211572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1371/2741] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:18 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:18.215658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.226637 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fba18-ebd3-47e4-b097-93b7603436de admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.267764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:18.284535 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.285208 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.285493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.285797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.286489 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.288404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.370046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1371/2742] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:18 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:18.374297 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.398507 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d73a-fb9e-4c36-a447-bc5f48d61d65 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.403484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.404248 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.404603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.405079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.405944 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.407601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.418409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1372/2743] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 14:42:18.432492 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.443463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08958d1e-9fb9-41a4-a8d5-c6d3325bde53 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.448423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.449137 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.449435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.449983 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.450974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.469031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08436f6-031e-4f23-97ad-e76bed08fda2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.469031 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1372/2744] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:18.472106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.482296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4b3b-9e36-4415-857d-0d63c54c391f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.487029 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.488468 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.488758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.489039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] PATH_INFO: `/v3/projects/d8d4276594f442df93409e095fcb0ab0/users/d71cd90ce5f34c0ca008ab48d213227b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.506161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d4276594f442df93409e095fcb0ab0, user_id=d71cd90ce5f34c0ca008ab48d213227b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.508830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.594508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1373/2745] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:18 2022] PUT /identity/v3/projects/d8d4276594f442df93409e095fcb0ab0/users/d71cd90ce5f34c0ca008ab48d213227b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:18.605043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.605342 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.605620 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.633678 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] MFA Rules not processed for user `d71cd90ce5f34c0ca008ab48d213227b`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:18.635036 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:18.701153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1373/2746] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:18.705658 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.767830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510734-fcf3-4b8e-83b2-6d38a1abea07 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.772595 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.773283 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.773576 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.773957 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.774664 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.776574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:18.879508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1374/2747] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:18 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:18.883770 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:18.894467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62510c9-4d14-4772-a02f-8ed753190ea2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:18.951393 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:18.952066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:18.952354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:18.952635 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:18.953339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:18.955271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.043797 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1374/2748] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:18 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:19.047704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.058926 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e193b8-9bac-4052-84fc-82fc0741b999 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.064223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.064882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.065195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.065473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.066281 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.067943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.087615 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1375/2749] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:19.090860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.102082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dbb3e-d81b-4651-b5f9-4a0238ad21bb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.106745 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.107623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.108327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.108327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] PATH_INFO: `/v3/projects/ea4019683c23436ba86f899b3fb9596c/users/0870a768a3404ed59b3b8d4abf72d230/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.123845 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4019683c23436ba86f899b3fb9596c, user_id=0870a768a3404ed59b3b8d4abf72d230, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.126481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.243720 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1375/2750] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:19 2022] PUT /identity/v3/projects/ea4019683c23436ba86f899b3fb9596c/users/0870a768a3404ed59b3b8d4abf72d230/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:42:19.247771 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.257697 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836019-7289-4e65-82ec-8a394522fe7f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.314835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.315780 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.316134 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.316340 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.317059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.318735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f64ed-1c2d-432b-9584-b17620b8dd7e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.328315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1376/2751] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:19.333013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.343617 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79121f-bb83-42ba-a12e-7b3d6d08c4b4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.349177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.350085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.350372 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.350652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] PATH_INFO: `/v3/projects/ea4019683c23436ba86f899b3fb9596c/users/0870a768a3404ed59b3b8d4abf72d230/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.352656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4019683c23436ba86f899b3fb9596c, user_id=0870a768a3404ed59b3b8d4abf72d230, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.355326 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.365418 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1376/2752] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:19 2022] PUT /identity/v3/projects/ea4019683c23436ba86f899b3fb9596c/users/0870a768a3404ed59b3b8d4abf72d230/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:19.376747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.377040 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.377321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.404628 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] MFA Rules not processed for user `0870a768a3404ed59b3b8d4abf72d230`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:19.405922 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:19.422978 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:19.475964 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1377/2753] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:19.487694 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.489539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:19.495212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:19.549728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b01761-274b-47e9-bf0c-1fa59fc225eb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.553820 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.554649 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.555045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.555428 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.565000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.566455 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.576531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1377/2754] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:19.839508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:19.840910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:19.853589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4fdbd-a3e9-4831-be38-bd55cd2c2729 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:19.857366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:19.858050 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:19.858372 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:19.858656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:19.916102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:19.917533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:19.926948 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1378/2755] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:29.690075 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=25 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:29.922801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=49 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:31.600059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:31.600059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:31.665758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c187b9-7f91-400c-bcdc-9f4ebd3eff2b service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:31.670350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:31.671211 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:31.671496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:31.671772 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] PATH_INFO: `/v3/users/336a80f361934b8d8ed214883f652d60` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:31.672974 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336a80f361934b8d8ed214883f652d60)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:31.674961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:31.878875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336a80f361934b8d8ed214883f652d60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:31.890543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336a80f361934b8d8ed214883f652d60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:31.974213 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336a80f361934b8d8ed214883f652d60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:31.974834 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336a80f361934b8d8ed214883f652d60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:31.975652 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336a80f361934b8d8ed214883f652d60'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:31.989015 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1378/2756] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:42:31 2022] DELETE /identity/v3/users/336a80f361934b8d8ed214883f652d60 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:32.245867 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:32.259353 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3b76-ce52-4b60-a442-a7897c725d22 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:32.321851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:32.322941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:32.323229 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:32.324548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] PATH_INFO: `/v3/projects/35413c62675b457a84da91d845f96f2b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:32.325797 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35413c62675b457a84da91d845f96f2b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:32.327683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:32.461315 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35413c62675b457a84da91d845f96f2b', 'request_id': 'req-45da8827-3990-4680-b05f-aaad92eacff2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.467704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35413c62675b457a84da91d845f96f2b', 'request_id': 'req-45da8827-3990-4680-b05f-aaad92eacff2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.539896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35413c62675b457a84da91d845f96f2b', 'request_id': 'req-45da8827-3990-4680-b05f-aaad92eacff2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.551321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] The token cache is being invalidate because project 35413c62675b457a84da91d845f96f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:32.551612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.552236 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.553795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1379/2757] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:32 2022] DELETE /identity/v3/projects/35413c62675b457a84da91d845f96f2b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:32.557822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:32.559336 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:32.572891 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fbea-d3bd-40ab-9e5a-6405284627e1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:32.629156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:32.630068 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:32.630353 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:32.630631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] PATH_INFO: `/v3/users/13b54bda40e242b0ada8a63bde1d6ed2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:32.631944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b54bda40e242b0ada8a63bde1d6ed2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:32.633930 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:32.655901 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:32.960147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b54bda40e242b0ada8a63bde1d6ed2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:32.973912 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b54bda40e242b0ada8a63bde1d6ed2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.026539 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b54bda40e242b0ada8a63bde1d6ed2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.027053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b54bda40e242b0ada8a63bde1d6ed2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.027661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e918ea14-59d3-4028-b927-e86da55f4352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b54bda40e242b0ada8a63bde1d6ed2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.040799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1379/2758] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:42:32 2022] DELETE /identity/v3/users/13b54bda40e242b0ada8a63bde1d6ed2 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:33.331881 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:33.333338 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:33.349119 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8827-3990-4680-b05f-aaad92eacff2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:33.410586 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:33.411472 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.411765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.412048 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] PATH_INFO: `/v3/projects/09b9181d380e44009bc3f01b94dad329` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.413280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b9181d380e44009bc3f01b94dad329)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:33.415214 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:33.535879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:33.547973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b9181d380e44009bc3f01b94dad329', 'request_id': 'req-c730fe30-61ed-415a-96ef-3e9bf4db95cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.554051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b9181d380e44009bc3f01b94dad329', 'request_id': 'req-c730fe30-61ed-415a-96ef-3e9bf4db95cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.609822 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b9181d380e44009bc3f01b94dad329', 'request_id': 'req-c730fe30-61ed-415a-96ef-3e9bf4db95cf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.621651 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] The token cache is being invalidate because project 09b9181d380e44009bc3f01b94dad329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:42:33.621973 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.624264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c730fe30-61ed-415a-96ef-3e9bf4db95cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:42:33.624264 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1380/2759] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:42:33 2022] DELETE /identity/v3/projects/09b9181d380e44009bc3f01b94dad329 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:33.633933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.634414 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.634531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.653152 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:33.654390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:33.716662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1380/2760] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:33.724961 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.725257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.725533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.743995 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:33.745416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:33.753930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1381/2761] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:33.759000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:33.770082 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be784-d7dc-4497-9d7e-48958493be3f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:33.775592 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:33.776263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.776560 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.776840 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.777642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:42:33.778128 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:33.779900 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:33.792273 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1381/2762] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:42:33.802610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:33.812003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1ef71-1052-4b48-a98a-a9cf1eb8bec5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:33.817316 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:33.817997 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.818282 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.818558 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.819242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:33.821117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:33.897049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1382/2763] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:33 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:33.901407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:33.911228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c781e-3be7-43b8-afaf-1eb08406ed5d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:33.968266 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:33.968925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:33.969212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:33.969490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:33.970457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:33.972536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:34.025192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1382/2764] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:33 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:34.029329 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:34.040062 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3257eb-af95-43e1-becd-ecf1ba5062a9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:34.044727 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:34.051525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:34.051525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:34.051525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:34.051525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:34.051525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:34.059580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1383/2765] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:34.063856 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:34.074740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9467d6e-592f-4be6-a2ea-4b4330e819bf admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:34.080758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:34.085945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:34.085945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:34.085945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:34.085945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:34.085945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199af05-ad75-482b-a57c-85bde5348109 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:34.094905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1383/2766] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:34.099101 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:34.109052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99998b-9652-4381-bced-31b635901c12 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:34.114228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:34.115104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:34.115389 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:34.115699 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] PATH_INFO: `/v3/projects/4c597b767cad4ce288fa65d68c5eaff5/users/79cca3fb8e6840e688fddecdbce20f1c/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:34.131152 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c597b767cad4ce288fa65d68c5eaff5, user_id=79cca3fb8e6840e688fddecdbce20f1c, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:34.133932 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:34.238656 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1384/2767] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:34 2022] PUT /identity/v3/projects/4c597b767cad4ce288fa65d68c5eaff5/users/79cca3fb8e6840e688fddecdbce20f1c/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:34.243774 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef5246-0ef4-48aa-98fc-48ea9a479b96 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:34.244074 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef5246-0ef4-48aa-98fc-48ea9a479b96 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:34.252209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef5246-0ef4-48aa-98fc-48ea9a479b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:34.252209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:34.261888 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-96ef5246-0ef4-48aa-98fc-48ea9a479b96 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:34.263104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-96ef5246-0ef4-48aa-98fc-48ea9a479b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:34.321352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1384/2768] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:34.331195 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:34.335382 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:34.390904 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45693e45-0541-4dbb-a186-627a00f6a6e9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:34.394914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:34.404321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:34.404321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:34.404321 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:34.405456 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:34.406924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:34.417135 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1385/2769] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:39.693990 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:39.694451 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:39.694663 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:39.730200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] MFA Rules not processed for user `79cca3fb8e6840e688fddecdbce20f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:39.731618 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:39.827586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1385/2770] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:39.832318 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:39.849004 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baa933-8631-468a-8984-cdd3c1a3e896 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:39.854019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:39.854692 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:39.854986 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:39.855273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:39.855987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:39.857930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:39.953600 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1386/2771] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:42:39 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:42:39.953825 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:39.963868 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea84fc-ed1b-49cb-b793-fd654a1478a0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:40.022076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:40.022814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:40.023103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:40.023382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:40.024079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:40.026029 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:40.112465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1386/2772] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:42:39 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:42:40.116496 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:40.131830 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eacd2-e7d8-4c6b-8819-5d14a874f06a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:40.141352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:40.142554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:40.143024 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:40.143273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:40.144016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:40.145706 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:40.162672 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1387/2773] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:40.166587 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:40.181200 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfca7d-41cb-466e-ba14-a899055bd801 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:40.187873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:40.189064 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:40.189357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:40.189642 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] PATH_INFO: `/v3/projects/5a9de42952f449609995b6743220fd0c/users/283de7607447468fbb35e830141a35f4/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:40.206614 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9de42952f449609995b6743220fd0c, user_id=283de7607447468fbb35e830141a35f4, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:40.210009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:40.280382 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:40.285909 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1387/2774] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:40 2022] PUT /identity/v3/projects/5a9de42952f449609995b6743220fd0c/users/283de7607447468fbb35e830141a35f4/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:40.290122 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:40.300933 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8df1e-1f81-4036-9e03-9be539f8888e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:40.334452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:40.385227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:40.386272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:40.386720 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:40.387176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:40.388076 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:40.389914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ab0d-1599-4ba0-bf61-ad42f55ef4cf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:40.409352 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1388/2775] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:42:40.410509 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:40.422979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea021ff-c55e-4bbb-8a86-031099b0916c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:40.428442 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:40.429438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:40.429813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:40.430201 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] PATH_INFO: `/v3/projects/5a9de42952f449609995b6743220fd0c/users/283de7607447468fbb35e830141a35f4/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:40.432296 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9de42952f449609995b6743220fd0c, user_id=283de7607447468fbb35e830141a35f4, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:40.435280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:40.447141 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1388/2776] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:42:40 2022] PUT /identity/v3/projects/5a9de42952f449609995b6743220fd0c/users/283de7607447468fbb35e830141a35f4/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:42:46.062084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:46.062084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:46.062084 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:46.090148 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] MFA Rules not processed for user `283de7607447468fbb35e830141a35f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:42:46.091652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:46.183210 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1389/2777] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:42:46.199525 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:42:46.201265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:42:46.289580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4e3d6-9ec0-4a13-9c34-2025d6791f1f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:42:46.295288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:42:46.295952 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:42:46.296239 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:42:46.296517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:42:46.359591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:42:46.361244 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5e127-b4e5-4b98-9593-198b9e26f5c5 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:42:46.371112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1389/2778] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:42:56.179711 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=51 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:42:56.366833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:08.611567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:08.613093 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:08.686931 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdfd8f-f81e-4097-ae99-bb2347e36746 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:08.689402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:08.690293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:08.690574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:08.690848 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] PATH_INFO: `/v3/users/d71cd90ce5f34c0ca008ab48d213227b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:08.692057 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d71cd90ce5f34c0ca008ab48d213227b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:08.694543 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:08.729841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c7a-7bda-48ae-b779-1b05e06639eb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:08.730385 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c7a-7bda-48ae-b779-1b05e06639eb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:08.730517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c7a-7bda-48ae-b779-1b05e06639eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:08.748212 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bfb46c7a-7bda-48ae-b779-1b05e06639eb None None] MFA Rules not processed for user `79cca3fb8e6840e688fddecdbce20f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:08.749501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfb46c7a-7bda-48ae-b779-1b05e06639eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:08.757166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1390/2779] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:43:08.902489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd71cd90ce5f34c0ca008ab48d213227b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:08.903147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd71cd90ce5f34c0ca008ab48d213227b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:08.978700 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd71cd90ce5f34c0ca008ab48d213227b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:08.979416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd71cd90ce5f34c0ca008ab48d213227b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:08.991500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd71cd90ce5f34c0ca008ab48d213227b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:09.007742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1390/2780] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:08 2022] DELETE /identity/v3/users/d71cd90ce5f34c0ca008ab48d213227b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:09.012260 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:09.014446 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:09.014742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:09.033870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:09.035145 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:09.073086 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:09.074540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:09.098330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1391/2781] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:43:09.108108 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:09.109543 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:09.141670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8633b9-553c-4803-b0ad-abc237c938e8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:09.145294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:09.145975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:09.146266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:09.146546 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:09.148410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:09.158202 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:09.159618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:09.165391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6778d50-d86e-46dc-944d-bb4a83b48ac5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:09.167905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1391/2782] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:43:09.170971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:09.171619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:09.171906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:09.172185 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:09.182253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:09.183683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:09.193683 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1392/2783] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:43:09.762898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:09.762898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:09.774669 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5445-d872-465e-b9bb-2f5f01756b4a service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:09.779430 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:09.780296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:09.780589 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:09.780913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] PATH_INFO: `/v3/projects/d8d4276594f442df93409e095fcb0ab0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:09.782150 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d4276594f442df93409e095fcb0ab0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:09.784262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:09.804104 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:09.960075 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d4276594f442df93409e095fcb0ab0', 'request_id': 'req-677aff68-36ab-481e-903c-d7a8a62d7e2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:09.967575 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d4276594f442df93409e095fcb0ab0', 'request_id': 'req-677aff68-36ab-481e-903c-d7a8a62d7e2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.045910 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d4276594f442df93409e095fcb0ab0', 'request_id': 'req-677aff68-36ab-481e-903c-d7a8a62d7e2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.055442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] The token cache is being invalidate because project d8d4276594f442df93409e095fcb0ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:10.056117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.057102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.061872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1392/2784] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:09 2022] DELETE /identity/v3/projects/d8d4276594f442df93409e095fcb0ab0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:10.063118 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:10.071872 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:10.073399 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:10.075957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5ebb7-7168-4027-9240-32c943442b57 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:10.144619 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:10.145538 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:10.146532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:10.146814 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] PATH_INFO: `/v3/users/0870a768a3404ed59b3b8d4abf72d230` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:10.148018 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0870a768a3404ed59b3b8d4abf72d230)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:10.150021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:10.156220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aff68-36ab-481e-903c-d7a8a62d7e2d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:10.161572 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:10.162250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:10.162540 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:10.162816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:10.164573 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:10.176326 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:10.177965 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:10.187734 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1393/2785] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:43:10.287995 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:10.292951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0870a768a3404ed59b3b8d4abf72d230'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.305411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0870a768a3404ed59b3b8d4abf72d230'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.357012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0870a768a3404ed59b3b8d4abf72d230'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.357700 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0870a768a3404ed59b3b8d4abf72d230'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.358423 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c23fddf7-a10b-4cf4-87aa-434095777f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0870a768a3404ed59b3b8d4abf72d230'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:10.371957 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1393/2786] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:10 2022] DELETE /identity/v3/users/0870a768a3404ed59b3b8d4abf72d230 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:10.936408 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:10.942286 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:10.952602 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951ca9-761f-4156-9003-84cb21808d55 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.016310 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.017278 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.017561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.017865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] PATH_INFO: `/v3/projects/ea4019683c23436ba86f899b3fb9596c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.019049 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4019683c23436ba86f899b3fb9596c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.021010 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.186222 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:11.194800 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4019683c23436ba86f899b3fb9596c', 'request_id': 'req-9ea881e6-30ac-41c3-9259-4a49249775c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:11.200951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4019683c23436ba86f899b3fb9596c', 'request_id': 'req-9ea881e6-30ac-41c3-9259-4a49249775c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:11.373402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4019683c23436ba86f899b3fb9596c', 'request_id': 'req-9ea881e6-30ac-41c3-9259-4a49249775c0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:11.384063 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] The token cache is being invalidate because project ea4019683c23436ba86f899b3fb9596c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:11.384351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:11.384975 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9ea881e6-30ac-41c3-9259-4a49249775c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:11.386531 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1394/2787] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:10 2022] DELETE /identity/v3/projects/ea4019683c23436ba86f899b3fb9596c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:11.391411 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.391921 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.392037 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.410558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:11.412362 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:11.474971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1394/2788] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:11.482696 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.482985 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.483265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.501051 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:11.502268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:11.510513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1395/2789] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:11.521602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.531673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a521a35-9364-4fc7-a0ff-07357690ebcc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.536971 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.537624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.538022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.538347 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.539085 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:43:11.539399 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.541042 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.552915 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1395/2790] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:43:11.557537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.566974 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bc127-8cfb-4842-9ac1-2eec9047534f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.573723 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.577938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.577938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.577938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.577938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.578364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.688802 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1396/2791] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:11 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:11.693380 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.704330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4593-304d-4632-8bef-81da8feeebf6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.766292 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.767140 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.767387 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.767665 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.768354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.770274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.892839 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1396/2792] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:11 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:11.913158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.923534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0a91-8c99-42d5-ae41-b9b7cc728e69 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.928511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.929162 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.929450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.929729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.930484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.932071 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.944262 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1397/2793] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:11.948435 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.961933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341266-c8fe-4984-8066-0b4efd6d1709 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.963583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.964236 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.964524 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.964807 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:11.965591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:11.967227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337c54-3ec0-4c8f-a550-fa112beb5514 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:11.977549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1397/2794] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:11.981618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:11.992221 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f00e3-591e-44b5-9fb5-e1a6f62a98f6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:11.996862 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:11.997742 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:11.998056 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:11.998335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] PATH_INFO: `/v3/projects/c9b767ad39a246b1832e8780ad5a3981/users/87577ded304e4677a64e83e8c9f879e2/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.018816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b767ad39a246b1832e8780ad5a3981, user_id=87577ded304e4677a64e83e8c9f879e2, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.021525 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.099298 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1398/2795] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:11 2022] PUT /identity/v3/projects/c9b767ad39a246b1832e8780ad5a3981/users/87577ded304e4677a64e83e8c9f879e2/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:43:12.110357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.110646 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.110919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.137751 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] MFA Rules not processed for user `87577ded304e4677a64e83e8c9f879e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:12.138973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:12.148138 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:12.202831 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1398/2796] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:12.207293 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.271840 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ec9f-1c9d-473d-a6cc-ed71151cbd0a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.276781 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.277452 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.277746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.278058 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.278792 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.281322 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.444415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1399/2797] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:12 2022] POST /identity/v3/projects => generated 369 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:12.448496 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.459255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c254-c043-431f-be63-b79b39996db5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.514898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.515566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.515875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.516139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.516832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.518792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.607542 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1399/2798] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:12 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:12.608930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.620328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b8eb-d836-4120-b9d5-f5fae76fb78f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.625275 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.626005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.626300 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.631539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.631539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.631539 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.640874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1400/2799] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:12.652026 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.663877 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b6c0d-b87d-4fc8-ba91-fc0da3cd512c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.670675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.671906 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.672219 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.672491 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] PATH_INFO: `/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c/users/e2ebdddd9e124328878c02393ed40e5a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.695419 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac33b832c2a646c1b9eccff1f1ac3f1c, user_id=e2ebdddd9e124328878c02393ed40e5a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.700257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.860526 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1400/2800] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:12 2022] PUT /identity/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c/users/e2ebdddd9e124328878c02393ed40e5a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:12.864446 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.874640 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32cb52-bafc-40fd-adde-1331585bed5d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.908633 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:12.933827 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.934481 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.938287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.938287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.938287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.938287 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f43dc1-e3de-44d0-8b48-d22e981bac69 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.947654 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1401/2801] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:12.951899 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:12.963600 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44032be-605f-40cd-b657-c222cf569e62 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:12.969312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:12.970246 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:12.972723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:12.972723 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] PATH_INFO: `/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c/users/e2ebdddd9e124328878c02393ed40e5a/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:12.973066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac33b832c2a646c1b9eccff1f1ac3f1c, user_id=e2ebdddd9e124328878c02393ed40e5a, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:12.976849 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:12.989895 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1401/2802] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:12 2022] PUT /identity/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c/users/e2ebdddd9e124328878c02393ed40e5a/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:43:13.001508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:13.001508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:13.001508 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:13.031941 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] MFA Rules not processed for user `e2ebdddd9e124328878c02393ed40e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:13.033277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:13.130387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1402/2803] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:13.135456 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:13.137300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:13.196985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb944-69af-432d-aafc-600df8337a11 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:13.200578 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:13.201300 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:13.201584 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:13.201890 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:13.211744 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:13.213449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ccab-a077-4cda-aa22-b5fbbcfedc4c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:13.230144 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1402/2804] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:16.899569 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:16.968410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6808eb5-550e-40be-ad8b-a1d67d5c5c64 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:16.973043 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:16.973951 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:16.974238 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:16.974518 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] PATH_INFO: `/v3/users/87577ded304e4677a64e83e8c9f879e2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:16.975707 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87577ded304e4677a64e83e8c9f879e2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:16.977630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:17.200020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87577ded304e4677a64e83e8c9f879e2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:17.200705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87577ded304e4677a64e83e8c9f879e2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:17.292841 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87577ded304e4677a64e83e8c9f879e2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:17.293453 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87577ded304e4677a64e83e8c9f879e2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:17.299156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:17.302864 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87577ded304e4677a64e83e8c9f879e2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:17.322448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1403/2805] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:16 2022] DELETE /identity/v3/users/87577ded304e4677a64e83e8c9f879e2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:43:17.327586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:17.328096 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:17.328209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:17.341046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:17.347602 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:17.350014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:17.420059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1403/2806] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:17.431184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:17.432667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:17.503345 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc541cd1-6d8c-4bbe-94f2-a39b75ca9b4e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:17.508393 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:17.509059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:17.509415 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:17.509704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:17.528729 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:17.531153 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:17.545903 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1404/2807] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:18.044701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:18.046216 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:18.059071 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3ca2b-2476-4370-97b7-72a16d3e699a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:18.064695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:18.065696 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:18.066022 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:18.066302 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] PATH_INFO: `/v3/projects/c9b767ad39a246b1832e8780ad5a3981` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:18.067636 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b767ad39a246b1832e8780ad5a3981)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:18.069996 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:18.215516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b767ad39a246b1832e8780ad5a3981', 'request_id': 'req-c9aa5a6e-4b38-4877-8651-720880b31c1a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.221966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b767ad39a246b1832e8780ad5a3981', 'request_id': 'req-c9aa5a6e-4b38-4877-8651-720880b31c1a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.228358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b767ad39a246b1832e8780ad5a3981', 'request_id': 'req-c9aa5a6e-4b38-4877-8651-720880b31c1a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.327090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] The token cache is being invalidate because project c9b767ad39a246b1832e8780ad5a3981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:18.327346 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.327925 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.329507 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1404/2808] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:18 2022] DELETE /identity/v3/projects/c9b767ad39a246b1832e8780ad5a3981 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:18.334028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:18.344319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5799-1cdd-4d5c-9534-2a9c0d6dd456 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:18.410842 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:18.412355 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:18.412855 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:18.413144 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] PATH_INFO: `/v3/users/e2ebdddd9e124328878c02393ed40e5a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:18.414448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ebdddd9e124328878c02393ed40e5a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:18.416570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:18.686976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ebdddd9e124328878c02393ed40e5a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.687874 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ebdddd9e124328878c02393ed40e5a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.797199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ebdddd9e124328878c02393ed40e5a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.797896 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ebdddd9e124328878c02393ed40e5a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.806384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71b8bf58-d5c5-44be-abb9-a93b2fc291e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ebdddd9e124328878c02393ed40e5a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:18.824484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:18.829760 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1405/2809] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:18 2022] DELETE /identity/v3/users/e2ebdddd9e124328878c02393ed40e5a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:19.390654 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:19.392132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:19.405790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5a6e-4b38-4877-8651-720880b31c1a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:19.466095 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:19.466969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:19.467257 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:19.467532 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] PATH_INFO: `/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:19.469014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac33b832c2a646c1b9eccff1f1ac3f1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:19.471014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:19.590427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:19.603062 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac33b832c2a646c1b9eccff1f1ac3f1c', 'request_id': 'req-43150c44-13bd-4bfa-8637-fc8f030a57ff'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:19.609184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac33b832c2a646c1b9eccff1f1ac3f1c', 'request_id': 'req-43150c44-13bd-4bfa-8637-fc8f030a57ff'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:19.614722 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac33b832c2a646c1b9eccff1f1ac3f1c', 'request_id': 'req-43150c44-13bd-4bfa-8637-fc8f030a57ff'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:19.716785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] The token cache is being invalidate because project ac33b832c2a646c1b9eccff1f1ac3f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:19.725920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:19.725920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-43150c44-13bd-4bfa-8637-fc8f030a57ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:19.725920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1405/2810] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:19 2022] DELETE /identity/v3/projects/ac33b832c2a646c1b9eccff1f1ac3f1c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:19.729305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:19.729924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:19.729924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:19.749898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:19.751250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:19.820448 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1406/2811] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:19.829092 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:19.829500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:19.829973 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:19.855764 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:19.858702 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:19.867927 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1406/2812] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:19.871296 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:19.882537 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d400-ef52-4190-bd4b-e37ac64c5a2f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:19.888070 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:19.888738 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:19.889025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:19.889305 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:19.890078 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:43:19.890391 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:19.891993 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:19.905421 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1407/2813] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:43:19.909940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:19.919545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04920dc-a726-4515-b631-ae9c9052e44a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:19.924339 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:19.924992 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:19.925274 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:19.925551 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:19.926725 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:19.928689 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:20.032363 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1407/2814] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:19 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:20.036788 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:20.047722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63890-199f-4de8-a512-0bdf0ad048b0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:20.119458 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:20.120067 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.120499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.120967 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.122127 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:20.127564 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:20.231039 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1408/2815] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:20 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:20.235583 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:20.249656 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a280-85c8-4141-9a00-1104ac529c3f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:20.257429 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:20.259175 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.259474 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.259761 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.260510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:20.262156 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:20.274607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1408/2816] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:20.278765 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:20.291691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb033b-7ce1-4783-8a39-0d5268ae050d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:20.296495 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:20.297161 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.297450 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.297891 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.301567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:20.301567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f7625-9199-4786-841d-0b224c28969c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:20.314189 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1409/2817] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:20.319046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:20.329732 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665d07-1060-4a4c-a05d-3d7489d226fc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:20.334936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:20.335860 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.336147 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.336430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] PATH_INFO: `/v3/projects/286a45cd86f248f88b365dc4a58282db/users/cc56bca31db043ca9b331b2826db192b/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.353013 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=286a45cd86f248f88b365dc4a58282db, user_id=cc56bca31db043ca9b331b2826db192b, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:20.355701 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:20.427841 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1409/2818] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:20 2022] PUT /identity/v3/projects/286a45cd86f248f88b365dc4a58282db/users/cc56bca31db043ca9b331b2826db192b/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:20.493594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.494265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.494705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.532695 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] MFA Rules not processed for user `cc56bca31db043ca9b331b2826db192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:20.536173 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:20.597737 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:20.608175 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1410/2819] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:20.612872 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:20.698644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fa55f-9b26-4dae-8939-beacfda82a0b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:20.704253 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:20.704911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:20.705228 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:20.705506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:20.706461 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:20.708607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.071863 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1410/2820] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:20 2022] POST /identity/v3/projects => generated 387 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:21.076106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.092728 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5c3fa-4110-4efd-b663-cbf73d0e82e3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.164535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.165216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.165533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.165844 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.171025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.171025 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.261072 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1411/2821] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:21 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:21.267021 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.278038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb54e-e2aa-479c-af0e-d3644234a8ed admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.282769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.283426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.283707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.283989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.284716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.287093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.298007 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1411/2822] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:21.302089 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.312128 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449b9ec-133a-4e3a-a0cd-37c3fc562e57 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.317156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.318083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.318366 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.318652 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] PATH_INFO: `/v3/projects/d75738823f904b67aee42dcf51291312/users/6a571bc42a8a4096845be792eaae9783/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.335177 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75738823f904b67aee42dcf51291312, user_id=6a571bc42a8a4096845be792eaae9783, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.337958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.476963 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1412/2823] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:21 2022] PUT /identity/v3/projects/d75738823f904b67aee42dcf51291312/users/6a571bc42a8a4096845be792eaae9783/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:21.481432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.491501 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022416b-a07b-476d-a706-3cf6df5fdfc3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.520103 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:21.548473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.549112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.549465 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.549741 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.554753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.554753 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76c2fc-03be-4b3f-a64c-c6325c18e747 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.563490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1412/2824] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:21.571269 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.582138 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5321-29f9-498e-902c-e84e8522556b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.588714 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.589610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.594858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.594858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] PATH_INFO: `/v3/projects/d75738823f904b67aee42dcf51291312/users/6a571bc42a8a4096845be792eaae9783/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.594858 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75738823f904b67aee42dcf51291312, user_id=6a571bc42a8a4096845be792eaae9783, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.595636 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.608425 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1413/2825] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:21 2022] PUT /identity/v3/projects/d75738823f904b67aee42dcf51291312/users/6a571bc42a8a4096845be792eaae9783/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:21.619941 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.620441 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.620791 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.654160 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] MFA Rules not processed for user `6a571bc42a8a4096845be792eaae9783`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:21.655373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:21.742742 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1413/2826] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:21.755105 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.756839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:21.828972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1336cb-0994-43ab-ad91-80ac8ea4bb8b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.833799 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.834473 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.834757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.835181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.845698 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.847233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe77198-1245-491c-a8d4-227af087c998 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:21.857199 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1414/2827] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:21.886566 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:21.950776 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437507e-07b2-4b09-a719-6da3e7b2b556 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:21.955297 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:21.956164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:21.956448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:21.956728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] PATH_INFO: `/v3/users/cc56bca31db043ca9b331b2826db192b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:21.958586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc56bca31db043ca9b331b2826db192b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:21.960842 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:22.160310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc56bca31db043ca9b331b2826db192b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:22.166287 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:22.173289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc56bca31db043ca9b331b2826db192b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:22.288322 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc56bca31db043ca9b331b2826db192b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:22.289289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc56bca31db043ca9b331b2826db192b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:22.290265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc56bca31db043ca9b331b2826db192b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:22.309038 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1414/2828] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:21 2022] DELETE /identity/v3/users/cc56bca31db043ca9b331b2826db192b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:22.314796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:22.315277 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:22.315395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:22.338494 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:22.339754 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:22.394022 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:22.409885 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1415/2829] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:22.419999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:22.421404 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:22.480675 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fdbb-8dc5-4afc-a60c-375e79b9f1ea admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:22.484907 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:22.485555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:22.485951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:22.486231 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:22.495734 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:22.497210 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:22.507305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1415/2830] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:22.979136 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:22.980574 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:22.993748 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfebcf-eb0d-4f03-ab39-bf194f2423b6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:23.000182 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:23.001165 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:23.001455 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:23.002207 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] PATH_INFO: `/v3/projects/286a45cd86f248f88b365dc4a58282db` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:23.004596 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=286a45cd86f248f88b365dc4a58282db)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:23.005517 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:23.218717 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286a45cd86f248f88b365dc4a58282db', 'request_id': 'req-dde5c211-385d-4341-aa92-fac99ce0f63a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.225732 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286a45cd86f248f88b365dc4a58282db', 'request_id': 'req-dde5c211-385d-4341-aa92-fac99ce0f63a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.349388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286a45cd86f248f88b365dc4a58282db', 'request_id': 'req-dde5c211-385d-4341-aa92-fac99ce0f63a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.360580 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] The token cache is being invalidate because project 286a45cd86f248f88b365dc4a58282db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:23.360915 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.361550 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.363121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1416/2831] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:22 2022] DELETE /identity/v3/projects/286a45cd86f248f88b365dc4a58282db => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:23.367631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:23.377331 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7d77-743f-4e2d-aad9-624b783d040c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:23.432699 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:23.433563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:23.433878 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:23.434159 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] PATH_INFO: `/v3/users/6a571bc42a8a4096845be792eaae9783` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:23.435471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a571bc42a8a4096845be792eaae9783)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:23.437394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:23.626790 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a571bc42a8a4096845be792eaae9783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.639427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a571bc42a8a4096845be792eaae9783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.727979 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a571bc42a8a4096845be792eaae9783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.732638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a571bc42a8a4096845be792eaae9783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.732638 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9508ec50-6317-4ad9-94df-9d2f29a046bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a571bc42a8a4096845be792eaae9783'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:23.734323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:23.743945 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1416/2832] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:23 2022] DELETE /identity/v3/users/6a571bc42a8a4096845be792eaae9783 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:24.510989 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:24.512489 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:24.525911 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5c211-385d-4341-aa92-fac99ce0f63a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:24.584001 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:24.584875 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:24.586420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:24.586420 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] PATH_INFO: `/v3/projects/d75738823f904b67aee42dcf51291312` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:24.586671 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75738823f904b67aee42dcf51291312)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:24.589412 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:24.721886 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75738823f904b67aee42dcf51291312', 'request_id': 'req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:24.730061 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75738823f904b67aee42dcf51291312', 'request_id': 'req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:24.847461 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75738823f904b67aee42dcf51291312', 'request_id': 'req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:24.859186 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] The token cache is being invalidate because project d75738823f904b67aee42dcf51291312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:24.875464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:24.875464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d0193b9-1c7f-4d86-aad0-fac1df6870e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:24.875464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1417/2833] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:24 2022] DELETE /identity/v3/projects/d75738823f904b67aee42dcf51291312 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:24.875832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:24.875832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:24.875832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:24.889735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:24.891009 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:24.952054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1417/2834] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:24.960702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:24.960996 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:24.961280 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:24.968007 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:24.992938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:24.992938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:24.992938 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1418/2835] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:24.994305 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.004262 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f04808-9f36-457f-a380-cf6024842ef5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.008961 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.009624 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.009944 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.010225 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.010999 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:43:25.011310 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.012919 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.026463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1418/2836] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:43:25.037566 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.047102 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d02a2-be74-4689-9b05-f86f69edb03a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.052028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.052686 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.052976 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.053249 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.054003 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.055914 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.220055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1419/2837] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:25 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:25.224554 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.234851 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab3920-935d-4eea-a2fd-bda26286deb0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.296721 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.297401 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.298415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.298415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.298692 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.300581 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.411933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1419/2838] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:25 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:25.412857 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.423395 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824a25d-2eab-4e92-974f-10272235ecb7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.428601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.429266 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.429553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.429871 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.430606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.432194 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.444077 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1420/2839] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:25.448271 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.458688 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256d88-66c4-44da-bb7a-49eb647d2a81 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.463755 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.465494 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.466070 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.466357 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.468117 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.470398 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5a9e6-5887-4be5-9866-d1995b183478 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.484293 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1420/2840] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:25.488096 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.499971 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e867843-4a33-4eed-931c-2d91a331eaa0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.506801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.508204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.508204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.508511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] PATH_INFO: `/v3/projects/ace4a2208d4f42f585a2298da6ddca70/users/d2e61d68a3084cd98fe523071de46c63/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.525582 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace4a2208d4f42f585a2298da6ddca70, user_id=d2e61d68a3084cd98fe523071de46c63, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.528493 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.606747 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1421/2841] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:25 2022] PUT /identity/v3/projects/ace4a2208d4f42f585a2298da6ddca70/users/d2e61d68a3084cd98fe523071de46c63/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:43:25.618615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.618911 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.619192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.646830 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] MFA Rules not processed for user `d2e61d68a3084cd98fe523071de46c63`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:25.648079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:25.696568 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:25.715444 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1421/2842] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:25.728206 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:25.729984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:25.817407 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e87ac-3957-4978-8307-4a6c5a0fcf24 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:25.821146 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1235df33fd6f4917850cf0f2aa850b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:25.821819 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:25.822110 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:25.822387 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:25.831544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:25.833005 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:25.842223 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1422/2843] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:27.251594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:27.253800 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:27.341965 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c382f-9054-4cfe-987d-5bd51720a630 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:27.351202 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:27.352309 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:27.352792 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:27.353143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:27.356784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:27.376227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:27.378487 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:27.394337 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1422/2844] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:30.452082 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:30.453499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:30.516772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb89fe-e83a-4b49-924b-f0005f1749b2 service glance] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:30.520484 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:30.521147 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:30.521435 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:30.521716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:30.523614 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:30.559374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:30.589865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:30.591319 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4059bbe-c4f7-4f4a-b6c7-633b5fd2d4b6 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:30.601716 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1423/2845] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:37.388976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=39 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:40.670561 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=13 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:43.447164 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:43.518224 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfb27b-1fec-4e0d-b992-dbbc2376877d service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:43.523733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:43.528438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:43.528438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:43.528438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] PATH_INFO: `/v3/users/d2e61d68a3084cd98fe523071de46c63` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:43.528438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e61d68a3084cd98fe523071de46c63)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:43.528438 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:43.873374 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e61d68a3084cd98fe523071de46c63'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:43.886493 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e61d68a3084cd98fe523071de46c63'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.001653 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e61d68a3084cd98fe523071de46c63'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.002321 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e61d68a3084cd98fe523071de46c63'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.002936 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e61d68a3084cd98fe523071de46c63'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.016805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1423/2846] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:43 2022] DELETE /identity/v3/users/d2e61d68a3084cd98fe523071de46c63 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:44.022181 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:44.022667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:44.022816 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:44.040554 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:44.041808 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:44.112553 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1424/2847] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:44.122437 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:44.123883 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:44.183445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c811-4a9f-4f8a-8f04-fae1ebb394da admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:44.187670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:44.188350 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:44.188670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:44.188956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:44.204017 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:44.205500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94d0cc-791e-4cb9-9a5b-27244884520d service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:44.219594 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1424/2848] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:44.704625 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:44.706254 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:44.718761 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030dc93-314a-4f0d-9572-208b743c3711 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:44.723676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:44.724557 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:44.724879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:44.725166 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] PATH_INFO: `/v3/projects/ace4a2208d4f42f585a2298da6ddca70` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:44.726460 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace4a2208d4f42f585a2298da6ddca70)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:44.728410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:44.883560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace4a2208d4f42f585a2298da6ddca70', 'request_id': 'req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.890220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace4a2208d4f42f585a2298da6ddca70', 'request_id': 'req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:44.994215 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace4a2208d4f42f585a2298da6ddca70', 'request_id': 'req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:45.006139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] The token cache is being invalidate because project ace4a2208d4f42f585a2298da6ddca70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:45.007750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:45.007750 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba0ba2e3-f62d-4b8b-9c8e-7b08413a8802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:45.008787 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1425/2849] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:44 2022] DELETE /identity/v3/projects/ace4a2208d4f42f585a2298da6ddca70 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:45.013661 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.014193 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.014312 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.033267 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:45.035054 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:45.059799 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:45.106555 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1425/2850] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:45.114384 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.116719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.116719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.134234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:45.135535 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:45.143813 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1426/2851] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:45.148180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.157956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34dcdf-14f6-4db7-951e-8d793875cfb5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.163595 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.164249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.164536 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.164817 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.165549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:43:45.165886 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.167483 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.179712 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1426/2852] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 14:43:45.184059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.193500 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2815a-e407-46b5-a475-a1b9bde15e9d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.198265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.198912 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.199197 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.199474 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.200170 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.202123 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.289158 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1427/2853] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:45.292498 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.303824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f6c4-827e-4b75-bc04-5ed48f292120 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.373576 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.374276 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.374563 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.374844 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.375582 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.377589 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.509690 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1427/2854] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:45.515233 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.530106 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef28ff-89c7-4284-ab66-cfeb4fbb33da admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.530909 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.531609 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.531901 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.532184 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.532924 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.535160 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.546627 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1428/2855] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:45.551220 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.561407 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4adc8-f8fb-4a2f-8f11-a158a5fcc569 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.566985 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.567658 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.568061 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.568352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.569093 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.570823 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b7146-d657-41c3-b2cc-d33fa455986b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.585184 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1428/2856] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:45.589532 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.600367 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d6db-525d-4c5d-9fff-8256ab6f92bd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.606079 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.606265 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.606416 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.606709 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] PATH_INFO: `/v3/projects/96ba89fe8891423db256356273b23802/users/e5654da82f9c4db58b5877349a7d507f/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.622274 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ba89fe8891423db256356273b23802, user_id=e5654da82f9c4db58b5877349a7d507f, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.624954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:45.733606 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1429/2857] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:45 2022] PUT /identity/v3/projects/96ba89fe8891423db256356273b23802/users/e5654da82f9c4db58b5877349a7d507f/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:45.745989 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.746288 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.749166 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.773235 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] MFA Rules not processed for user `e5654da82f9c4db58b5877349a7d507f`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:45.774500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:45.841014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1429/2858] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:45.845701 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:45.867631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:45.910534 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c200-0a0c-4a4e-bb04-e7c2800ab3a2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:45.915736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:45.916402 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:45.916691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:45.916972 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:45.917683 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:45.919660 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.055055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1430/2859] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:45 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:46.059245 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.071187 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b8c8d-a5eb-4f56-a5f1-48e92b846899 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.085903 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:46.133918 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.134681 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.134976 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.135265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.136058 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.138104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.254098 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1430/2860] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:46 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:43:46.258220 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.268928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafccf54-8949-4950-ae70-3a203dd297aa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.273757 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.274464 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.274756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.275038 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.275803 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.277410 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.288273 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1431/2861] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:46.292575 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.303631 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82b4c4-82fc-4faa-9b7f-a8c7b2ecdc75 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.308417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.309315 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.309605 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.309920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] PATH_INFO: `/v3/projects/da1cad711305446a823cd92ffac6a6ed/users/94e597012f5d48baa04595e5de6142a3/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.326354 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1cad711305446a823cd92ffac6a6ed, user_id=94e597012f5d48baa04595e5de6142a3, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.329047 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.414055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1431/2862] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:46 2022] PUT /identity/v3/projects/da1cad711305446a823cd92ffac6a6ed/users/94e597012f5d48baa04595e5de6142a3/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:46.417984 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.429216 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04aa6-020a-41f5-824f-78b635803761 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.493033 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.493705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.494031 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.494308 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.495044 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.496655 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2abbbe-2776-4a6c-bcea-d55e7730d6e0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.506798 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1432/2863] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:46.511050 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.521687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951684-712e-45dc-a379-2bb647b9fc04 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.526373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.527289 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.527580 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.527859 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] PATH_INFO: `/v3/projects/da1cad711305446a823cd92ffac6a6ed/users/94e597012f5d48baa04595e5de6142a3/roles/df6ee0fb73ff4f50bbefafd179e47864` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.529951 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1cad711305446a823cd92ffac6a6ed, user_id=94e597012f5d48baa04595e5de6142a3, role_id=df6ee0fb73ff4f50bbefafd179e47864)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.532603 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.544083 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1432/2864] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:46 2022] PUT /identity/v3/projects/da1cad711305446a823cd92ffac6a6ed/users/94e597012f5d48baa04595e5de6142a3/roles/df6ee0fb73ff4f50bbefafd179e47864 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:46.554970 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.556954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.556954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.582443 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] MFA Rules not processed for user `94e597012f5d48baa04595e5de6142a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:46.583703 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:46.665735 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1433/2865] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:46.677601 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:46.680356 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:46.758640 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868f8c7-3c19-4ea1-b34e-9b375259a752 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:46.762334 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:46.762981 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:46.763263 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:46.763541 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:46.828227 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:46.829677 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142fc6d-87da-4c0a-968a-c20e09db89f2 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:46.839757 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1433/2866] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:50.108877 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:50.131928 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:50.193839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647578a-ad9c-4a86-adb0-8316cc8b7248 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:50.199424 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:50.200522 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:50.200837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:50.201114 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] PATH_INFO: `/v3/users/e5654da82f9c4db58b5877349a7d507f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:50.202511 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5654da82f9c4db58b5877349a7d507f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:50.205954 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:50.454250 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5654da82f9c4db58b5877349a7d507f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:50.454821 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5654da82f9c4db58b5877349a7d507f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:50.551234 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5654da82f9c4db58b5877349a7d507f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:50.552047 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5654da82f9c4db58b5877349a7d507f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:50.560418 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5654da82f9c4db58b5877349a7d507f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:50.580204 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1434/2867] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:50 2022] DELETE /identity/v3/users/e5654da82f9c4db58b5877349a7d507f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:50.585511 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:50.586132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:50.586132 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:50.606360 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:50.607984 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:50.638272 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:50.683486 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1434/2868] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 14:43:50.694905 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:50.696530 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:50.774426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b066847-bda1-410c-873b-9113e3277854 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:50.779705 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:50.780702 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:50.781159 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:50.781618 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:50.793882 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:50.795371 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:50.806432 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1435/2869] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:51.302634 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:51.305445 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:51.324655 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd575-bda4-4911-8ebb-6b9a8fe908a2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:51.333659 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:51.335238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:51.335756 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:51.336265 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] PATH_INFO: `/v3/projects/96ba89fe8891423db256356273b23802` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:51.338615 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ba89fe8891423db256356273b23802)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:51.342775 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:51.465358 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ba89fe8891423db256356273b23802', 'request_id': 'req-9436b6a3-fc5f-4e41-80db-de9c0431212a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.472670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ba89fe8891423db256356273b23802', 'request_id': 'req-9436b6a3-fc5f-4e41-80db-de9c0431212a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.479759 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ba89fe8891423db256356273b23802', 'request_id': 'req-9436b6a3-fc5f-4e41-80db-de9c0431212a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.571896 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] The token cache is being invalidate because project 96ba89fe8891423db256356273b23802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:51.572255 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.573139 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.575012 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1435/2870] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:51 2022] DELETE /identity/v3/projects/96ba89fe8891423db256356273b23802 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:51.579739 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:51.593312 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a933b7-bf53-48d1-b08f-2e523d73b0a7 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:51.644257 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:51.673533 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:51.674637 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:51.674921 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:51.675228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] PATH_INFO: `/v3/users/94e597012f5d48baa04595e5de6142a3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:51.676513 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e597012f5d48baa04595e5de6142a3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:51.678499 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:51.905365 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e597012f5d48baa04595e5de6142a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.905955 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e597012f5d48baa04595e5de6142a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.978679 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e597012f5d48baa04595e5de6142a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.979514 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e597012f5d48baa04595e5de6142a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:51.988597 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7950b0b2-bbee-4ba3-89cc-4b9d799f90e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e597012f5d48baa04595e5de6142a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.010020 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1436/2871] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:43:51 2022] DELETE /identity/v3/users/94e597012f5d48baa04595e5de6142a3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:52.548956 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:52.551604 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:52.565569 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436b6a3-fc5f-4e41-80db-de9c0431212a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:52.642787 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:52.643709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:52.643972 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:52.644251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] PATH_INFO: `/v3/projects/da1cad711305446a823cd92ffac6a6ed` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:52.645564 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1cad711305446a823cd92ffac6a6ed)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:52.647540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:52.820975 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1cad711305446a823cd92ffac6a6ed', 'request_id': 'req-1b5db69e-534d-4811-8d39-370d4e8f008d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.822457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:52.829066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1cad711305446a823cd92ffac6a6ed', 'request_id': 'req-1b5db69e-534d-4811-8d39-370d4e8f008d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.836427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1cad711305446a823cd92ffac6a6ed', 'request_id': 'req-1b5db69e-534d-4811-8d39-370d4e8f008d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.939033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] The token cache is being invalidate because project da1cad711305446a823cd92ffac6a6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:43:52.939328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.940059 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1b5db69e-534d-4811-8d39-370d4e8f008d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:43:52.942010 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1436/2872] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:43:52 2022] DELETE /identity/v3/projects/da1cad711305446a823cd92ffac6a6ed => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:52.948354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:52.948940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:52.948940 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:52.968879 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:52.970176 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:53.034796 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1437/2873] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:53.042873 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.043167 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.043453 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.063416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:53.064662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:53.074033 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1437/2874] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:53.078670 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.089988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cdc9-cd63-4217-8068-9b5c9a0e969f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.095000 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.095676 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.095962 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.096243 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.097604 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 14:43:53.097982 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.099641 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.112756 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1438/2875] 10.222.0.93 () {58 vars in 1152 bytes} [Thu Jul 14 14:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 14:43:53.117447 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.127785 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201ecdd-26b5-4372-884f-709661143cc3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.132788 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.133463 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.133758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.134065 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.134769 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.136718 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.231163 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1438/2876] 10.222.0.93 () {60 vars in 1152 bytes} [Thu Jul 14 14:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 14:43:53.235987 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.246733 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81972-f3b9-498e-97a5-f987d35c621e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.299601 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:43:53.325560 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.326242 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.326559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.327027 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.327647 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.329605 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.462411 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1439/2877] 10.222.0.93 () {60 vars in 1143 bytes} [Thu Jul 14 14:43:53 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 14:43:53.467846 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.480478 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135808-b19e-4202-84d2-cda0c5cd1683 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.485328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.486040 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.486330 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.486611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.487409 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.489264 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.501123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1439/2878] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:53.506241 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.519930 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad1844-0a11-4d22-9ffa-6420073b02aa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.526981 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.528095 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.528607 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.529116 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.530570 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.533758 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41289b4c-f876-4d11-b989-8a9c49b32225 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.556117 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1440/2879] 10.222.0.93 () {58 vars in 1121 bytes} [Thu Jul 14 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 14:43:53.560687 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.571940 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28728b39-e2d0-45d3-a816-4c94746537db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.577709 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.579077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.579531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.580000 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] PATH_INFO: `/v3/projects/39fed467abc64dc4931449d9c332fffe/users/a152b2998ccc45e5b28f952959ac0592/roles/1302304811c14a18bc5c38884ac0ed1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.599504 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39fed467abc64dc4931449d9c332fffe, user_id=a152b2998ccc45e5b28f952959ac0592, role_id=1302304811c14a18bc5c38884ac0ed1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.602464 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.731833 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1440/2880] 10.222.0.93 () {60 vars in 1482 bytes} [Thu Jul 14 14:43:53 2022] PUT /identity/v3/projects/39fed467abc64dc4931449d9c332fffe/users/a152b2998ccc45e5b28f952959ac0592/roles/1302304811c14a18bc5c38884ac0ed1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:43:53.737839 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7bf4-4a0f-4975-8b24-73e95b96cf37 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.739542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7bf4-4a0f-4975-8b24-73e95b96cf37 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.739542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7bf4-4a0f-4975-8b24-73e95b96cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.758130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c96e7bf4-4a0f-4975-8b24-73e95b96cf37 None None] MFA Rules not processed for user `2be999d55fd24e54ae049877c17b8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:53.759375 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c96e7bf4-4a0f-4975-8b24-73e95b96cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:53.839386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1441/2881] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:53.851818 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:53.853607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:53.930113 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca12d-8d32-4b96-9ada-4a78a151c699 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:53.942390 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:53.943143 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:53.943391 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:53.943673 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:53.954477 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:53.956001 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:53.968323 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1441/2882] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:55.588246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135a44-b2dd-44ab-9490-4e6b81b4ea0b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:55.588814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135a44-b2dd-44ab-9490-4e6b81b4ea0b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:55.588814 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135a44-b2dd-44ab-9490-4e6b81b4ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:55.629966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-de135a44-b2dd-44ab-9490-4e6b81b4ea0b None None] MFA Rules not processed for user `a152b2998ccc45e5b28f952959ac0592`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:55.632704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-de135a44-b2dd-44ab-9490-4e6b81b4ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:55.735562 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1442/2883] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:55.744716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:55.746613 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:55.818832 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47abd98-3ac9-4b9c-a5ef-d9ab642d9599 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:55.822531 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '398d1954cb704f3898d3984f5c6102a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:55.823209 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:55.823500 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:55.823784 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:55.834198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:55.836716 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:55.852043 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1442/2884] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:43:57.755623 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:57.756230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:57.756230 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:57.779441 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] MFA Rules not processed for user `a152b2998ccc45e5b28f952959ac0592`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:43:57.781268 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:57.792354 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1443/2885] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:43:58.063490 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:43:58.064598 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:58.077740 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506013f8-8251-4491-b404-0fa7b3a89921 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:43:58.082557 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:43:58.083249 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:43:58.083586 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:43:58.083874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:43:58.086055 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:43:58.098695 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:43:58.100170 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33001424-f610-46ff-8734-cee4f41a05b1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:43:58.110352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1443/2886] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:44:04.467442 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:44:04.468876 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:44:04.523218 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:44:04.544794 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852383d7-54a9-4eff-9b28-7aeb1c925b3a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:44:04.551098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:44:04.551098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:44:04.551386 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:44:04.551635 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:44:04.554279 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:44:04.567869 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:44:04.569465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0f96e-fb2a-4482-a268-4aee3deba491 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:44:04.580401 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1444/2887] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:44:08.105549 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:44:14.596098 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=11 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:44:20.559920 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:44:20.560426 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:44:20.560523 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:44:20.577611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] MFA Rules not processed for user `a152b2998ccc45e5b28f952959ac0592`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:44:20.578850 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:44:20.585623 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1444/2888] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:44:30.572558 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:44:43.811795 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5248550-c998-43e4-b3cc-2cade9390abb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:44:43.812350 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5248550-c998-43e4-b3cc-2cade9390abb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:44:43.812488 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5248550-c998-43e4-b3cc-2cade9390abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:44:43.862361 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c5248550-c998-43e4-b3cc-2cade9390abb None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:44:43.863603 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5248550-c998-43e4-b3cc-2cade9390abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:44:43.933364 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1445/2889] 10.222.0.93 () {58 vars in 1009 bytes} [Thu Jul 14 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:44:44.113628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:44:44.115076 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:44:44.210122 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd46e-949e-4909-86fc-f5034253b63c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:44:44.215308 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:44:44.215955 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:44:44.216242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:44:44.216522 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:44:44.233910 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:44:44.235556 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:44:44.246406 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1445/2890] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:44:53.930187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=40 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:44:54.241517 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=41 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:45:00.685958 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:00.686567 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:00.687396 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:00.712630 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] MFA Rules not processed for user `1391f1de60b9465295b94c16f8a54872`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:45:00.713898 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:45:00.721719 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1446/2891] 10.222.0.93 () {58 vars in 967 bytes} [Thu Jul 14 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:45:00.734153 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:45:00.745014 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05d6c-1126-4ea1-8967-8c856475b7ed service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:45:00.750752 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d4c712ae2b04fc68eed213213452664', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:45:00.751394 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:00.751684 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:00.751966 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:00.760460 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:45:00.762079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:45:00.770933 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1446/2892] 10.222.0.93 () {60 vars in 1384 bytes} [Thu Jul 14 14:45:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 14:45:04.000866 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:45:04.008645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3badf00-f410-429a-895d-3c0c4fa8a9cc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:45:04.013465 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:45:04.014388 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:04.014675 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:04.014953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] PATH_INFO: `/v3/users/a152b2998ccc45e5b28f952959ac0592` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:04.016736 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a152b2998ccc45e5b28f952959ac0592)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:45:04.018859 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:45:04.312193 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a152b2998ccc45e5b28f952959ac0592'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.313026 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a152b2998ccc45e5b28f952959ac0592'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.416646 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a152b2998ccc45e5b28f952959ac0592'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.417328 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a152b2998ccc45e5b28f952959ac0592'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.426019 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-63cbc4c1-f537-4596-9b5b-436513601583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a152b2998ccc45e5b28f952959ac0592'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.444409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1447/2893] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:45:03 2022] DELETE /identity/v3/users/a152b2998ccc45e5b28f952959ac0592 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:45:04.709887 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:45:04.722352 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c6a2c-2487-47d3-83bf-5de16a516f96 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:45:04.786149 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:45:04.788758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:04.788758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:04.788758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] PATH_INFO: `/v3/projects/39fed467abc64dc4931449d9c332fffe` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:04.788758 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39fed467abc64dc4931449d9c332fffe)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:45:04.790674 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:45:04.955783 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39fed467abc64dc4931449d9c332fffe', 'request_id': 'req-c1127646-712b-4f10-8693-1a92324a6c92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.962670 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39fed467abc64dc4931449d9c332fffe', 'request_id': 'req-c1127646-712b-4f10-8693-1a92324a6c92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:04.968611 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39fed467abc64dc4931449d9c332fffe', 'request_id': 'req-c1127646-712b-4f10-8693-1a92324a6c92'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:05.046471 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] The token cache is being invalidate because project 39fed467abc64dc4931449d9c332fffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:45:05.046836 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:05.047559 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:45:05.049205 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1447/2894] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:45:04 2022] DELETE /identity/v3/projects/39fed467abc64dc4931449d9c332fffe => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:45:14.440722 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:45:15.042177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:45:35.170295 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:35.170837 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:35.170943 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:35.188849 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] MFA Rules not processed for user `79cca3fb8e6840e688fddecdbce20f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:45:35.190887 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:45:35.246028 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1448/2895] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:45:40.770898 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:45:40.772432 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:45:40.831427 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127646-712b-4f10-8693-1a92324a6c92 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:45:40.835165 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:45:40.835813 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:40.836104 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:40.836389 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:40.838545 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:45:40.910874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:45:40.912473 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:45:40.923481 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1448/2896] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:45:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:45:42.099157 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:45:42.108846 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1119e75-9b3e-433f-88c5-5c24bebbbbae None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:45:42.113055 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:45:42.113691 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:45:42.114015 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:45:42.114294 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:45:42.116068 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:45:42.130132 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:45:42.131548 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9e335-c753-4898-abdc-2ee292cf9417 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:45:42.142643 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1449/2897] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:45:50.918540 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=54 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:45:52.137704 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=31 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:46:58.540434 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:46:58.542053 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:46:58.553448 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339118-83d6-4614-a4cf-da91254d1c95 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:46:58.557417 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:46:58.558089 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:46:58.558373 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:46:58.558651 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:46:58.560516 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:46:58.571449 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:46:58.572958 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:46:58.582091 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1449/2898] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:47:08.577870 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:47:34.712130 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:47:34.712772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:47:34.712772 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:47:34.734374 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] MFA Rules not processed for user `79cca3fb8e6840e688fddecdbce20f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 14:47:34.735612 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:34.748246 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1450/2899] 10.222.0.93 () {58 vars in 957 bytes} [Thu Jul 14 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 14:47:41.762035 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:47:41.765462 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:41.783591 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3c33-f83a-4840-9846-9ea3f518c8e8 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:47:41.790251 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:47:41.793822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:47:41.793822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:47:41.793822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:47:41.793822 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:41.809650 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:47:41.811327 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:47:41.827425 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1450/2900] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:47:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:47:44.727667 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:47:46.927059 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:47:46.928542 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:46.945631 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2376-7439-4091-bcb0-460487e74a45 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:47:46.951351 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:47:46.952052 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:47:46.952357 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:47:46.952639 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:47:46.955035 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:46.966746 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:47:46.968439 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:47:46.978544 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1451/2901] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:47:51.821102 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:47:56.974272 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:47:57.718176 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:47:57.731280 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfdc4-741d-4880-a292-3384f7ba6f02 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:47:57.737180 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47007139f45e4e55b5a9fa2d2901ef9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:47:57.737882 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:47:57.738171 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:47:57.738457 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:47:57.740593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:47:57.755123 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:47:57.756733 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:47:57.769136 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1451/2902] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:48:07.763510 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:49:30.512835 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:30.514426 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:30.668335 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6b768-46b3-4d55-a3da-d1a1610c1f67 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:30.674187 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:30.674851 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:30.675139 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:30.675428 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:30.677333 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:30.690244 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:30.692083 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:30.703016 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1452/2903] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:49:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 14:49:40.698801 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=63 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:49:42.332192 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:42.354430 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077d9d-eaed-40f2-b7c9-3f7bd1c5f2c5 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:42.358905 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937f40c46a3a4c97897706bbc3425afb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:42.359593 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:42.359880 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:42.360158 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:42.362090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:42.432369 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:42.433748 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:42.445199 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1452/2904] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:49:46.857594 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:46.869470 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d43f5c-a95d-4887-9f32-ca0b7de722f1 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:46.876180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:46.876854 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:46.877141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:46.877422 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:46.879731 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:46.892755 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:46.894392 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:46.905451 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1453/2905] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:49:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:49:50.850969 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:50.852467 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:50.866608 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ea51-4123-49de-9809-c70dc4946fc2 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:50.871662 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:50.873090 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:50.873328 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:50.873607 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] PATH_INFO: `/v3/users/79cca3fb8e6840e688fddecdbce20f1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:50.874875 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cca3fb8e6840e688fddecdbce20f1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:50.876874 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:51.117077 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cca3fb8e6840e688fddecdbce20f1c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.130112 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cca3fb8e6840e688fddecdbce20f1c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.211779 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cca3fb8e6840e688fddecdbce20f1c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.212402 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cca3fb8e6840e688fddecdbce20f1c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.213045 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cca3fb8e6840e688fddecdbce20f1c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.227268 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1453/2906] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:49:50 2022] DELETE /identity/v3/users/79cca3fb8e6840e688fddecdbce20f1c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:49:51.528180 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:51.538751 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbf4c5-47f5-416d-8d54-e26b98311d0a service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:51.603873 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:51.604749 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:51.605034 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:51.605311 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] PATH_INFO: `/v3/projects/4c597b767cad4ce288fa65d68c5eaff5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:51.606611 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c597b767cad4ce288fa65d68c5eaff5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:51.608559 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:51.757103 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c597b767cad4ce288fa65d68c5eaff5', 'request_id': 'req-08f6bae1-eb17-43be-9ba7-64691e1db022'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.765141 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c597b767cad4ce288fa65d68c5eaff5', 'request_id': 'req-08f6bae1-eb17-43be-9ba7-64691e1db022'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.850121 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c597b767cad4ce288fa65d68c5eaff5', 'request_id': 'req-08f6bae1-eb17-43be-9ba7-64691e1db022'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.862251 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] The token cache is being invalidate because project 4c597b767cad4ce288fa65d68c5eaff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:49:51.862545 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.863211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:51.864913 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1454/2907] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:49:51 2022] DELETE /identity/v3/projects/4c597b767cad4ce288fa65d68c5eaff5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:49:51.869006 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:51.870506 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:51.884114 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30b121-5029-42cb-8a3d-a19070660f10 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:51.942980 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:49:51.949824 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:51.950885 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:51.951242 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:51.951597 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] PATH_INFO: `/v3/users/283de7607447468fbb35e830141a35f4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:51.952854 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283de7607447468fbb35e830141a35f4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:51.954924 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:52.169198 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283de7607447468fbb35e830141a35f4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.181707 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283de7607447468fbb35e830141a35f4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.292248 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283de7607447468fbb35e830141a35f4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.292805 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283de7607447468fbb35e830141a35f4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.293415 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283de7607447468fbb35e830141a35f4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.308046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1454/2908] 10.222.0.93 () {58 vars in 1223 bytes} [Thu Jul 14 14:49:51 2022] DELETE /identity/v3/users/283de7607447468fbb35e830141a35f4 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 14:49:52.654107 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:49:52.655610 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:49:52.671409 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6bae1-eb17-43be-9ba7-64691e1db022 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:49:52.742988 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be999d55fd24e54ae049877c17b8c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df24a39f5f0a46c8b40d27ec05417d32', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:49:52.743865 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:49:52.744156 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:49:52.744440 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] PATH_INFO: `/v3/projects/5a9de42952f449609995b6743220fd0c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:49:52.745661 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9de42952f449609995b6743220fd0c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:49:52.748289 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:49:52.770227 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:49:52.885645 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9de42952f449609995b6743220fd0c', 'request_id': 'req-c2495220-a770-4f05-aa57-c4733a937368'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:52.892211 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9de42952f449609995b6743220fd0c', 'request_id': 'req-c2495220-a770-4f05-aa57-c4733a937368'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:53.034953 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9de42952f449609995b6743220fd0c', 'request_id': 'req-c2495220-a770-4f05-aa57-c4733a937368'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:53.055966 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] The token cache is being invalidate because project 5a9de42952f449609995b6743220fd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 14:49:53.056228 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:53.065979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c2495220-a770-4f05-aa57-c4733a937368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 14:49:53.065979 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1455/2909] 10.222.0.93 () {58 vars in 1232 bytes} [Thu Jul 14 14:49:52 2022] DELETE /identity/v3/projects/5a9de42952f449609995b6743220fd0c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 14:50:02.589066 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:50:03.052824 dv-cinder-799871-30-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 14:55:00.124177 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 14:55:00.125628 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:55:00.196110 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec615a-6306-444d-ab00-c31d84b60864 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 14:55:00.256046 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768f4ab96bcf430f8e26b4d08d49581a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe18e4150846cb89e1e269df5dcd77', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 14:55:00.256728 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 14:55:00.257644 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 14:55:00.258416 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 14:55:00.260079 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 14:55:00.271843 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 14:55:00.273515 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea44287-15b9-4716-9dc2-04acb0ad8904 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 14:55:00.284238 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1455/2910] 10.222.0.93 () {60 vars in 1365 bytes} [Thu Jul 14 14:55:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 14:55:10.279735 dv-cinder-799871-30-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=58 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}